Select GPU by Model: How Much VRAM Do You Need to Run LLMs?

Parameter count → Memory requirement → Currently available GPUs and lowest hourly rates · Formula open for verification · Data updated: 2026-11-06

⚡ Direct answer: FP16 inference for 7–8B models requires ≈17GB VRAM (runs comfortably on a 24GB GPU); INT4-quantized 70–72B models require ≈42GB (80GB GPUs offer headroom), while FP16 full-precision needs 168GB (only single MI300X 192GB or B200 180GB fits); Choose GPUs by use case with the selection tool; compare compute-per-dollar rankings.

Parameter tier	Representative models (any within this tier)	Inference FP16/BF16	Inference INT8	Inference INT4	Full fine-tuning (reference)
1.5B	Qwen2.5-1.5B	≈4 GB	≈2 GB	≈1 GB	≈24 GB
7-8B	Llama-3.1-8B、Qwen2.5-7B	≈17 GB	≈9 GB	≈5 GB	≈112 GB
14B	Qwen2.5-14B	≈34 GB	≈17 GB	≈9 GB	≈224 GB
32B	Qwen2.5-32B	≈77 GB	≈39 GB	≈20 GB	≈512 GB
70-72B	Llama-3.1-70B、Qwen2.5-72B	≈168 GB	≈84 GB	≈42 GB	≈1120 GB

How is this calculated?

Inference VRAM ≈ parameter count × bytes per parameter × 1.2 (the 1.2 factor reserves headroom for KV cache and runtime overhead): FP16/BF16: 2 bytes per parameter; INT8: 1 byte; INT4: 0.5 byte.

Full fine-tuning (AdamW mixed-precision) ≈ parameter count × 16 bytes (weights + gradients + optimizer states combined);\n LoRA fine-tuning memory usage is close to inference, ~1.2–1.5× inference demand; multi-GPU memory adds up when single-GPU capacity is insufficient (communication efficiency calculated separately).

This is a rough estimate: long context lengths may inflate KV cache by several GB; actual memory usage also varies across frameworks and quantization implementations. Always validate via real-world testing before deployment—we provide formulas only, not guarantees of successful execution.

Currently available GPU models (recomputed per market cycle)

Select parameter scale and precision to list GPUs with sufficient VRAM and currently monitored lower hourly rental rates:

Required VRAM: 16.8 GB

GPU model	VRAM	Currently lower hourly rate	¥/TFLOPS·hour	Price Comparison Page

Interactive GPU selection requires JavaScript enabled; the quick-reference table above covers mainstream memory tiers, and real-time pricing per GPU model is available at homepage price comparison table.

Definition: ‘Currently lowest hourly rate’ in this table uses the same methodology as the homepage price comparison (real-time prices prioritized; expired prices for reference only);\n The ‘Full fine-tuning’ column gives a rough memory estimate only — it does not guarantee sufficient compute power. Training throughput also depends on bandwidth and interconnect; see per-unit compute ranking and quick specs overview per GPU model page.\n Representative models shown are common publicly available examples for this memory tier only — not recommendations; availability and stock status depend on the platform’s order page.