TechCompare LogoTechCompare

How much VRAM does MiMo-V2.5 310B (MoE) need at Q4_K_M? Xiaomi 1M SWA model

By TechCompare · Updated · How we check

MiMo-V2.5 310B at Q4_K_M is a 244 GB resident deployment for 1M context, and 174 GB of that is weights.

Calculator

Estimated VRAM required

244 GB

310B params at Q4_K_M, 1,048,576 token context, batch 1, inference.

Weights
174 GB
KV cache
48.3 GB
Overhead
22.2 GB

Estimate accuracy: Weights within ~2%. KV cache within ~5% for standard GQA models, ~10% for MLA or hybrid state estimates. Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.

Sliding-window attention applied: This model caps 4 of every 5 layers at a 128-token window. KV cache estimate is 81% smaller than naive full-attention math at this context length.

Hardware that fits

Apple entries show installed unified memory separately from a model budget based on an 8 GB reserve. That budget is a planning heuristic, not a fixed hardware limit.

Apple M3 Ultra 256GB
Unified
~248 GB
98% of estimate
NVIDIA B300
Datacenter
288 GB
85% used
Same 310B @ Q4_K_M, 1,048,576 context scenario, pre-loaded. Change the quant, shrink the context, or try a different preset.
Edit in the full tool

MiMo-V2.5 310B at Q4_K_M with native 1M context needs about 244 GB of VRAM with all experts resident. The smaller sibling of MiMo-V2.5-Pro activates 15B parameters per token from a 310B total pool. That doesn't shrink what has to stay loaded, because different tokens pick different experts. Sliding-window attention with a 128-token window keeps the key-value cache to 48 GB at the full 1M context.

How this is calculated

The 310B parameter pool requires 174 GB of weights at Q4_K_M. The 48-layer architecture uses group-query attention with variable KV heads: 8 in the 9 full-attention layers, 4 in the 39 sliding-window layers.

The QK and V dims share the same asymmetry as the Pro variant, so the effective head_dim is 160. Sliding-window attention caps 39 of 48 layers at a 128-token window. The key-value cache at the full 1M context uses about 48 GB. Activation and software overhead adds about 22 GB.

Verdict

The 310B is the practical self-hosting pick in the MiMo lineup. It fits 4 cards of 80 GB or 2 H200 cards of 141 GB with NVLink, while the Pro variant at 1.02T needs 687 GB resident and activates 42B parameters per token against 15B here.

More MiMo scenarios

Kimi K3.1 Pro at Q4_K_M
A 2.78T expert pool with 24 latent attention layers and fixed state in the other 69.
View details
Mistral Large 4 at Q4_K_M
A 1T expert pool with 63 latent cache owners and 44 layers that also store an index key.
View details
Muse Spark Open at Q4_K_M
Only 16 of its 62 layers grow with context, and the other 46 hold a 2,048 token window.
View details

Related guides

Context window explained
How a model context window caps prompt size.
View guide

Frequently asked questions

What's the difference between MiMo-V2.5 and MiMo-V2.5-Pro?
Same architecture family with asymmetric QK and V attention dims and sliding-window attention. The Pro variant has a 1.02T total pool with 42B active across 70 layers. The base variant has a 310B total pool with 15B active across 48 layers. The base is the practical pick if you have a small multi-GPU server or workstation.
Does MiMo-V2.5 fit on a single GPU?
Not on an 80GB card. Resident mode needs 244 GB, which takes 4 cards of 80 GB. The 15B active count doesn't shrink that, because different tokens pick different experts. Moving expert weights to system RAM cuts VRAM, but the host then has to hold them and every token gets slower.
Does MiMo-V2.5 310B fit on a single GPU?
Not in resident mode. The 310B parameter pool at Q4 is roughly 174 GB just for weights, exceeding any single GPU's memory. For high-QPS serving, multi-GPU tensor parallelism is the standard.
How does MiMo-V2.5 compare to the Mixtral-style MoEs for self-hosting?
The same architecture shape scaled up: more total experts, more parameters per token path, and stronger quality, at a weight footprint no single consumer GPU holds. If you've self-hosted Mixtral 8x7B, the MiMo-V2.5 story is the same trade tuned up a generation: you still need multi-GPU or very fast CPU-side offload, but the active-subset throughput per token is competitive.

Related tools

RAM Latency Calculator

Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.

Use tool

Power Cost Estimator

Estimate annual electricity costs for your PC, Server, or TV.

Use tool

Data Transfer Calculator

Estimate transfer times for files over USB, WiFi, Ethernet, and more.

Use tool

Memory and Storage Throughput Visualizer

Visualize the large speed gap between CPU cache, RAM, and storage.

Use tool