MiMo-V2.6-Pro VRAM requirements: about 744 GB at Q4 and 1M context

By TechCompare · Updated · How we check

MiMo-V2.6-Pro needs about 744 GB in this Q4_K_M, 1,048,576-token, batch-one scenario. The full 1.02T weight pool accounts for 623 GB.

Calculator

Estimated VRAM required

744 GB

1020B params at Q4_K_M, 1,048,576 token context, batch 1, inference.

Weights
623 GB
KV cache
53.8 GB
Overhead
67.6 GB

Estimate accuracy: Planning estimate. Rounded model counts, tensor formats, cache packing, and workspace can change actual allocation. Confirm the exact checkpoint and serving engine with a load test.

Sliding-window attention applied: This model caps 6 of every 7 layers at a 128-token window. KV cache estimate is 86% smaller than naive full-attention math at this context length.

MiMo-V2.6-Pro uses 10 global and 60 sliding attention layers. The cache uses 192-wide keys and 128-wide values. This budget covers the backbone and text cache. Vision/audio processing, speculative decoding, and backend workspace can need extra memory.

Hardware that fits

Unified memory machines show a model budget, not their installed memory. We hold back 8 GB for the system, or use the vendor's GPU limit when that's lower. It's a planning estimate, not a hard limit.

No single card or machine in our catalog has enough memory. The multi-GPU setups below do.

Multi-GPU setups

8× RTX PRO 6000 96GB
Pro · 93.9 GB per card
768 GB
98% used
4× MI300X
Datacenter · 187 GB per card
768 GB
98% used

Pooled cards show up once a model outgrows the biggest single card of that kind. Each card past the first keeps about 1 GB for its own buffers, and that's counted here. Your inference engine has to support the split, and speed also depends on how the cards connect.

MiMo-V2.6-Pro is a large MoE deployment even though each token activates about 42B parameters. This estimate keeps its full 1.02T parameter pool resident, uses Q4_K_M's planning factor of 0.61 bytes per parameter, and stores the text attention cache in FP16. The roughly 744 GB result includes a 10% runtime allowance. It isn't a measured checkpoint footprint or a guarantee that a particular quantization backend supports the architecture.

How this is calculated

Xiaomi's published configuration has 70 backbone layers with hidden size 6,144. Ten use global attention and 60 use a 128-token sliding window. Both groups have eight KV heads. Keys are 192 values wide and values are 128, so the cache calculation uses their combined width of 320 rather than assuming identical key and value dimensions.

At 1,048,576 tokens, the global layers plus sliding buffers consume about 53.73 GB in FP16. Add 622.20 GB for Q4 weights and 67.60 GB of modeled overhead to reach 743.52 GB, or about 692.46 GiB.

Reducing context to 32,768 tokens cuts the cache to 1.72 GB and the total to about 686.31 GB. These calculations use the RL/MOPD backbone layout and rounded published parameter count. They don't separately predict image or audio activation peaks, speculative draft buffers, communication workspace, or a runtime's prefill allocation strategy.

Verdict

Plan MiMo Pro around a large memory pool, and check the real checkpoint before you choose hardware. A shorter context helps, but it can't remove the 623 GB weight budget. Eight 80 GB GPUs give 640 GB in total, below this full context estimate before uneven sharding is counted.

You'd need more cards, lower weight precision or an offload setup your engine supports. A memory fit alone doesn't show that throughput will be acceptable or that multi GPU execution works.

More MiMo scenarios

Kimi K3.1 Pro at Q4_K_M
A 2.78T expert pool with 24 latent attention layers and fixed state in the other 69.
View details
Mistral Large 4 at Q4_K_M
A 1T expert pool with 63 latent cache owners and 44 layers that also store an index key.
View details
Muse Spark Open at Q4_K_M
Only 16 of its 62 layers grow with context, and the other 46 hold a 2,048 token window.
View details

Frequently asked questions

Why can't I size MiMo Pro as a 42B model?
The 42B figure describes parameters activated for a token. Routing can choose different experts on the next token, so the full pool still needs storage. The calculator's all-resident mode budgets all 1.02T parameters. An estimate based on activated parameters is a mathematical scenario, not an implementation of expert streaming. A real offload plan must name which tensors stay in GPU memory and where the rest live.
Does sliding attention make the 1M cache constant size?
Only the 60 local layers have a bounded cache. The ten global layers retain attention state across the whole sequence, so their memory still grows with context and concurrent sequences. At 1M context they dominate the modeled 53.73 GB text cache. Switching to a supported 8-bit cache roughly halves that cache budget, but it doesn't halve weights or total memory.
Is Q4_K_M the format of Xiaomi's download?
No. Q4_K_M here supplies a consistent approximate weight budget for comparing precision choices. Check the publisher's actual checkpoint, a conversion's tensor formats, and the serving engine's model support before using the number as a purchasing target. Mixed precision, scale tensors, unquantized modules, and optional components can change the resident allocation. A download's size also doesn't include runtime memory.
Do the RL and MOPD variants need different backbone presets?
Their published backbone dimensions and attention layout match, so the same architecture estimate applies. Post-training differences can change behavior without changing these memory dimensions. Keep the exact checkpoint revision in a deployment record and inspect its configuration when converting weights. This page uses the MOPD configuration as its linked reference and doesn't predict quality differences between the checkpoints.
What should I measure after the model loads?
Measure resident weights, cache allocation, and peak prompt-processing memory separately. Test the intended context limit and number of simultaneous sequences, then include representative image or audio inputs if you'll serve them. The 10% allowance is a planning heuristic, not a cap on those peaks. Also check each GPU's allocation, because an aggregate fit can still fail on the most heavily loaded shard.