How much VRAM does Nemotron 3.5 Lightning 30B need at Q4_K_M? NVIDIA's single-card 1M model
By TechCompare · Updated · How we check
Nemotron 3.5 Lightning is a 29 GB resident model at the full 1M context, so a 48 GB pro card runs it with all experts loaded. Shorter contexts shrink the cache and the total.
Calculator
Estimated VRAM required
28.4 GB
31.6B params at Q4_K_M, 1,048,576 token context, batch 1, inference.
Estimate accuracy: Weights within ~3% of most GGUF files from Q4_K_M up, ~7% for Q3_K_M and Q2_K. KV cache within ~5% for standard GQA models, ~10% for MLA or hybrid state estimates. Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.
Hardware that fits
Unified memory machines show a model budget, not their installed memory. We hold back 8 GB for the system, or use the vendor's GPU limit when that's lower. It's a planning estimate, not a hard limit.
Nemotron 3.5 Lightning 30B at Q4_K_M needs 29 GB of VRAM with all experts resident at the full 1M context. NVIDIA's hybrid Mamba-2 plus transformer design keeps the cache small, so a 1M-context model fits a single 48 GB card. It uses 3B active parameters per token, but that doesn't shrink what has to stay loaded, because different tokens pick different experts.
How this is calculated
Only 6 attention layers carry a key-value cache. 23 of the other blocks are Mamba-2 state-space layers with a fixed-size recurrent state, 0.05 GB at batch one. Weights at Q4_K_M are 19.3 GB, the KV cache with 2 KV heads at head_dim 128 across 6 attention layers is 6.5 GB at 1M, and overhead adds 2.7 GB.
Verdict
Choose Lightning when you want a 1M window on one card: it totals 29 GB resident, so a single 48 GB card handles it outright. Nemotron 3 Ultra 550B, the other hybrid, needs 384 GB at the same context.
More Nemotron scenarios
Related guides
Frequently asked questions
How can a 1M-context model fit in 29 GB?
Nemotron 3.5 Lightning or gpt-oss 20B for a local agent?
Related tools
RAM Latency Calculator
Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.
Use toolPower Cost Estimator
Estimate annual electricity costs for your PC, Server, or TV.
Use toolData Transfer Calculator
Calculate file transfer time or the speed you need over USB, Wi-Fi or Ethernet, with latency.
Use toolMemory and Storage Throughput Visualizer
Visualize the large speed gap between CPU cache, RAM, and storage.
Use tool