How much VRAM does Mixtral 8x7B need at Q4_K_M? MoE memory math
Mixtral 8x7B is the rare model where memory is the bottleneck and compute is cheap. It needs at minimum a 32 GB card or a 24 GB card with Q8 KV cache and reduced context. When it fits, it's one of the fastest high-quality local options.
Mixtral 8x7B at Q4_K_M with 32K context needs about 34 GB of VRAM. Mixture-of-experts is the wrinkle here: even though only 2 of the 8 experts activate per token, all 8 must be in memory at all times. That makes Mixtral 8x7B effectively a 47B-parameter model from a memory standpoint, even though it computes like a 12B.
By TechCompare · Updated
Calculator
Estimated VRAM required
33.7 GB
47B params at Q4_K_M, 32,768 token context, batch 1, inference.
Estimate accuracy: Weights within ~2%. KV cache within ~5% for standard GQA models, ~10% for MLA or hybrid state estimates. Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.
Custom architecture - SWA not applied. If you're modeling Gemma 3/4 or Mistral Nemo, pick the preset for accurate KV cache.
Hardware that fits
Apple entries show installed unified memory separately from a model budget based on an 8 GB reserve. That budget is a planning heuristic, not a fixed hardware limit.
Just barely too small
How this is calculated
MoE is a compute optimization, not a memory one. The 8 expert FFN blocks plus the shared attention give Mixtral roughly 47B total parameters that must all be loaded in VRAM. At Q4_K_M that's about 26 GB of weights, plus ~4.3 GB of KV cache at 32K context. The benefit is throughput - inference runs at the speed of a 12B dense model, so you get 70B-class quality at 12B-class generation speed if it fits.
Verdict
Mixtral is the case study for MoE memory reality: only 2 of 8 experts run per token, but the router can hit any expert on any forward pass, so all 47B of parameters must stay resident. At Q4_K_M that's 26 GB of weights plus 4.3 GB of KV cache at 32K context, totaling about 34 GB. A 32 GB card fits it cleanly, a 24 GB card needs Q8 KV and a context cap to clear the gate, otherwise the same gen-rate math applies. The reward is throughput that matches a 12-13B dense model because only 2 experts compute per token, so you get 47B-class quality at 13B-class generation speed. That's the structural MoE win that justifies the memory cost.
More Mistral scenarios
Related guides
Frequently asked questions
Why does Mixtral 8x7B use so much memory if only 2 experts run?
How fast is Mixtral 8x7B compared to a dense 47B model?
Why does a Mixture-of-Experts model still need all parameters in memory?
Related tools
RAM Latency Calculator
Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.
Use tool ➜Power Cost Estimator
Estimate annual electricity costs for your PC, Server, or TV.
Use tool ➜Data Transfer Calculator
Estimate transfer times for files over USB, WiFi, Ethernet, and more.
Use tool ➜Memory and Storage Throughput Visualizer
Visualize the massive speed difference between CPU cache, RAM, and storage.
Use tool ➜