文章总结： 该文档对DeepSeek-R1蒸馏模型(1.5B至70B)在不同精度(FP16/INT8/INT4)下的VRAM占用与相对准确率进行了基准测试。核心结论是量化可显著降低显存需求但会牺牲精度;在有限VRAM下,使用高量化的大模型(如14BINT4)比低量化的小模型(如7BFP16)性价比更高。文档提供了具体GPU配置建议,例如16GB显存适合运行14BINT4模型,并验证了Ollama默认使用INT4量化的合理性。 综合评分： 87 文章分类： AI安全,安全工具,技术标准

The Ultimate DeepSeek-R1 VRAM Cheat Sheet: Benchmarking 1.5B to 70B Models

原创

衡水铁头哥 衡水铁头哥

铁军哥

2026年7月7日 07:42 北京

在小说阅读器读本章

去阅读

Previously（一道四年级数学题，DeepSeek-R1的32b以下模型全军覆没，视频为证！）, we briefly compared the capabilities across the DeepSeek-R1 distilled models (from 1.5B up to 32B). You might be wondering: why is there such a massive difference in performance? I recently dug deeper into the mechanics behind this, and here is what I found.

Essentially, every parameter in a Large Language Model (LLM) is a number. The default storage precision is typically BF16 or FP16. As any programmer knows, FP16 stands for 16-bit half-precision floating-point, meaning each data value takes up 16 bits, or 2 bytes. Based on this math, a 7B (7 billion parameter) model file should be around 14 GB. In reality, it matches up perfectly: it’s about 15,237,852,832 bytes (14.1 GB). When you run this model, the VRAM (GPU memory) required is roughly identical—taking up about 14,623 MB (14.28 GB).

What about BF16 (16-bit Brain Floating Point)? While it occupies the exact same storage footprint as FP16, it sacrifices some fractional precision to offer a much wider dynamic range.

This is where INT8 and INT4 quantization come in. By using scaling factors, quantization compresses 16-bit data down to 8 bits (1 byte) or 4 bits (half a byte). This drastically reduces the model’s footprint. For example, after INT8 quantization, a 7B model shrinks to 8,098,524,832 bytes (7.54 GB). Because quantization involves complex mapping rather than simple zip-style compression, it’s slightly larger than exactly half the original size. However, when you load this quantized model into your GPU, factors like framework overhead and KV cache kick in. As a result, the actual VRAM usage exceeds the file size, hitting around 8300 MB (8.1 GB).

Take the default 7B model pulled by Ollama as another example. It uses INT4 quantization out of the box. The file size is 4,683,073,184 bytes (4.36 GB)—again, slightly larger than half of the INT8 version. Once loaded onto the GPU, the actual VRAM consumption reaches 5303 MB (5.18 GB).

When evaluating these models, the community loves to focus on relative accuracy—specifically, how these quantized versions score across various benchmarks compared to the full-blooded, unquantized 671B base model. I haven’t benchmarked the 671B locally (for obvious hardware reasons!), but according to DeepSeek’s official data: the relative accuracy of the 7B FP16 model is 44%, INT8 drops to 42%, and INT4 is 38%.

For the smallest 1.5B model, INT4 quantization drags relative accuracy down to a mere 22%. You need at least the INT4-quantized 14B model to hit the 60% “passing grade”. To break the 80% threshold, you have to step up to the 32B model with INT8 quantization, which achieves 82%.

Looking at the data in this table, the results from the math benchmarks we ran previously make total sense.

Aside from some performance overlap between the 7B and 8B models, the rest of the lineup strictly adheres to the rule of thumb: larger parameter counts inherently dominate. Of course, with evolving distillation architectures, the real-world “feel” and usability might yield unexpected surprises.

However, from a VRAM efficiency perspective, highly quantized versions of larger models often deliver better accuracy and ROI than lower-quantized smaller models. Let’s compare running the 7B model in FP16 versus the 14B model in INT4. The 14B INT4 model reduces VRAM usage by 27.7% while boosting relative accuracy by 36.4%. Talk about getting more bang for your buck!

Let’s map this to common GPU specs on the market:

A 16 GB card (like the Tesla T4) is arguably the sweet spot for running the 14B INT4 model.

If you step up to 24 GB of VRAM (like a Tesla P40, RTX 3090/4090, or A10), the 32B INT4 model is your best bet.

Even with a 32 GB Tesla V100, the 32B INT4 model remains the most practical and efficient choice.

Want to push it further? Running the 32B model in INT8 requires roughly 38 GB of VRAM, meaning you’d need a 48 GB GPU setup. But once you hit 48 GB of VRAM, you can just manage to squeeze in the 70B INT4 model (which consumes 47.5 GB), though you might experience bottlenecks without a bit more memory headroom.

Currently, the highest-tier cloud instances readily available typically feature quad-V100 setups, yielding 128 GB of VRAM. This is just enough to comfortably run the 70B INT8 model, which eats up 79 GB of VRAM while delivering a stellar 89% relative accuracy. If you’re chasing that 90%+ accuracy mark, you’ll need at least 141 GB of VRAM, which pushes you into the realm of high-end bare-metal servers.

Validation Details (Actual VRAM usage via nvidia-smi):

1、deepseek-r1:1.5b-qwen-distill-q4_K_M uses 1651 MB.

2、deepseek-r1:1.5b-qwen-distill-q8_0 uses 2281 MB.

3、deepseek-r1:1.5b-qwen-distill-fp16 uses 3661 MB.

4、deepseek-r1:7b-qwen-distill-q4_K_M uses 5303 MB.

5、deepseek-r1:8b-llama-distill-q4_K_M uses 6182 MB.

6、deepseek-r1:7b-qwen-distill-q8_0 uses 8300 MB.

7、deepseek-r1:8b-llama-distill-q8_0 uses 9385 MB.

8、deepseek-r1:14b-qwen-distill-q4_K_M uses 10577 MB.

9、deepseek-r1:7b-qwen-distill-fp16 uses 14623 MB.

10、deepseek-r1:8b-llama-distill-fp16 uses 16096 MB.

11、deepseek-r1:14b-qwen-distill-q8_0 uses 16609 MB.

12、deepseek-r1:32b-qwen-distill-q4_K_M uses 21450 MB.

13、deepseek-r1:14b-qwen-distill-fp16 uses 29234 MB.

14、deepseek-r1:32b-qwen-distill-q8_0 uses 38852 MB.

15、deepseek-r1:70b-llama-distill-q4_K_M uses 48654 MB.

16、deepseek-r1:32b-qwen-distill-fp16 uses 67446 MB.

17、deepseek-r1:70b-llama-distill-q8_0 uses 79106 MB.

All things considered, it makes perfect sense why Ollama sets INT4 quantized models as their default. The overall cost-to-performance ratio is exceptionally high for the average user. Of course, after seeing all this data, if you are running on a laptop with only 8 GB of unified memory or VRAM, your best and most reliable option is still the INT4 quantized 7B or 8B models.

What about your local setup? Which DeepSeek model are you currently running, and what GPU hardware is powering it? Let’s discuss in the comments below! If you found this benchmark breakdown helpful, make sure to follow for more deep dives into LLM deployment and infrastructure engineering.

***推荐阅读***

我们的WireGuard管理系统支持手机电脑了！全平台终端配置，支持扫码连接，一键搞定

保姆级教程：一条命令部署OpenVPN管理系统V4版，支持Win/Mac/安卓/iOS全平台接入

成本省下99.7%！用40元的腾讯云服务器自建IPsecVPN，成功对接企业级飞塔防火墙

终极进化：当swanctl遇上FRR，让你的Linux加密隧道化身SD-WAN雏形

流量指哪打哪！手把手教你用静态Segment List玩转SRv6流量工程

嫌SRv6报文太胖跑不动？带你在Ubuntu+FRR实战uSID微段压缩

22秒跑出密码！算力碾压再升级，揭秘WiFi6+WPA3的致命短板

嫌一键部署不过瘾？带你手搓Hermes智能体，主打一个通透

十倍性能提升！Ubuntu 26.04深度实测：当VPP遇上OpenVPN，带宽直接冲破 6.5Gbps！

性能暴涨670 %！当WireGuard遇上VPP，带宽直冲7.4 Gbps！

手机也能跑DeepSeek-R1/Qwen3了：零成本搭建AI推理平台

2048卡昇腾910C集群算力集群交付工程手册

2048卡H100算力中心100G无阻塞存储网建设方案

免责声明：

本文所载程序、技术方法仅面向合法合规的安全研究与教学场景，旨在提升网络安全防护能力，具有明确的技术研究属性。

任何单位或个人未经授权，将本文内容用于攻击、破坏等非法用途的，由此引发的全部法律责任、民事赔偿及连带责任，均由行为人独立承担，本站不承担任何连带责任。

本站内容均为技术交流与知识分享目的发布，若存在版权侵权或其他异议，请通过邮件联系处理，具体联系方式可点击页面上方的联系我。

本文转载自：铁军哥 衡水铁头哥 衡水铁头哥《The Ultimate DeepSeek-R1 VRAM Cheat Sheet: Benchmarking 1.5B to 70B Models》