LLM VRAM Calculator
Estimate the GPU memory needed to run or fine-tune a large language model. Pick a preset or enter your own values.
Estimated VRAM required
15.0 GB
9B params at Q4_K_M, 262,144 token context, batch 1, inference.
Estimate accuracy: Weights within ~2%. KV cache within ~5% for standard GQA models, ~10% for MLA or hybrid state estimates. Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.
KV cache exceeds model weights: Consider lowering the context length to save on VRAM. Contexts between 8K and 64K are generally more typical for local setups.
Hybrid linear attention applied: 24 of 32 layers use fixed recurrent state, so only 8 full-attention layers contribute to the context-growing KV cache. The fixed state adds about 0.05 GB at batch 1 and does not grow with context length.
Hardware that fits
Apple entries show installed unified memory separately from a model budget based on an 8 GB reserve. That budget is a planning heuristic, not a fixed hardware limit.
How to use this tool
Pick a model
Select a model from the dropdown (Llama 3.1, Qwen 2.5, Mistral, Mixtral, Gemma, DeepSeek, Phi-3, and more). The calculator loads that model's parameter count, layer count, and hidden size automatically.
Choose a quantization format
Pick a quantization level: FP16 (full precision, ~2 bytes/param), Q8_0 (~1 byte/param), Q5_K_M, or Q4_K_M (~0.56 bytes/param). Lower precision shrinks the weight memory at a small quality cost.
Set context length and batch size
Enter the context length in tokens (8K, 32K, 128K, etc.) and the batch size. The KV cache grows linearly with both, so long contexts can dominate the total VRAM at low quantization.
Read the VRAM estimate
The calculator returns total VRAM with a breakdown of weights, KV cache, and runtime overhead. It also lists which GPUs and unified-memory machines have enough memory to run that configuration.
About this tool
The LLM VRAM Calculator estimates how much GPU memory you need to run or fine-tune a large language model. Pick a popular model from the dropdown (Llama 3.1, Qwen 2.5, Mistral, Mixtral, Gemma, DeepSeek, Phi-3, and more), choose a quantization format, set the context length and batch size, and the calculator returns total VRAM along with a breakdown of weights, KV cache, and runtime overhead. It also lists which GPUs and unified-memory machines have enough memory.
Use it before downloading a model to confirm it fits, when shopping for hardware to run a specific size class, or when deciding whether quantization (Q4_K_M, Q5_K_M, Q8_0) buys you enough headroom to run a tier larger. The math behind it is the same as what llama.cpp and vLLM use internally, so the numbers match what you'll actually see when you load the model.
Formula
Total VRAM = weights + KV cache + activation overhead. Weights = params × bytes_per_param (FP16 = 2, Q8 ≈ 1, Q4_K_M ≈ 0.56). For standard full-context attention, KV cache = 2 × layers × KV_heads × head_dim × context × batch × KV_dtype_bytes. Models without separate KV-head metadata fall back to hidden_size, while sliding-window and cache-compression presets reduce the result when applicable. For training, use an architecture and batch-size specific budget. A common mixed-precision AdamW estimate is about 18 bytes per parameter before activations, compared with 2 bytes per parameter for FP16 inference. This page's training result is a planning estimate, not a full training-memory model.
When to use it
Ideal for choosing a model size that fits your existing GPU, planning a multi-GPU build for a specific model, or comparing quantization options before downloading multi-GB GGUF files. Pair with the Power Cost Estimator to factor running cost over the lifetime of an inference rig, and the RAM Latency Calculator if you're considering CPU offload as a fallback.
Quantization explained
Modern quantization (Q4_K_M, Q5_K_M, etc.) doesn't just lower precision uniformly. K-quants store most weights at the target bit width but keep a small fraction in higher precision (FP16 metadata, occasional Q6 blocks for important weights), which is why effective bytes-per-param numbers like 0.56 for Q4_K_M are slightly higher than the raw 4-bit value would suggest. The trade-off is favorable - you get most of the memory savings of pure 4-bit with quality very close to Q8_0 or FP16.
What we don't model
The calculator gives you a static memory ceiling. It doesn't capture transient memory spikes during long generations, batch scheduling overhead in serving engines, or the small reduction from techniques like prefix caching. The hardware list keeps installed Apple unified memory separate from an estimated model budget based on an 8 GB reserve. That budget is a planning heuristic, not an Apple-defined VRAM limit. Leave additional headroom for your actual macOS, apps, runtime, context, and batch workload to avoid out-of-memory errors.
Popular models and configurations
Pre-computed VRAM estimates for the model and quantization combinations visitors ask about most.
Frequently asked questions
How much VRAM do I need to run Llama 3.1 70B locally?
What's the difference between Q4_K_M, Q5_K_M, and Q8_0?
Why does context length affect VRAM so much?
What is KV cache and can I reduce it?
How much extra VRAM does training or fine-tuning need?
Can I split a model across multiple GPUs?
Does Apple unified memory count as VRAM?
Why doesn't my model fit even though the math says it should?
Related tools
LLM API Pricing Calculator
Compare API costs across major models (OpenAI, Anthropic, Google) with prompt caching.
Use tool ➜LLM Token Counter
Count tokens in any prompt for GPT, Claude, Gemini, and Llama with exact OpenAI tokenization.
Use tool ➜Power Cost Estimator
Estimate annual electricity costs for your PC, Server, or TV.
Use tool ➜RAM Latency Calculator
Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.
Use tool ➜Use this tool on your own site
Drop the LLM VRAM Calculator into your blog, docs, or CMS with one iframe snippet. Free, no signup. See all embeddable tools