TechCompare LogoTechCompare

How much VRAM does MiniMax-M3 428B (MoE) need at Q4_K_M? MiniMax 1M block-sparse

By TechCompare · Updated · How we check

MiniMax-M3 428B at Q4_K_M is a 405 GB resident deployment for 1M context. The full-resolution key-value cache is 129 GB of that, so context length is the lever you control.

Calculator

Estimated VRAM required

405 GB

428B params at Q4_K_M, 1,048,576 token context, batch 1, inference.

Weights
240 GB
KV cache
129 GB
Overhead
36.9 GB

Estimate accuracy: Weights within ~2%. KV cache within ~5% for standard GQA models, ~10% for MLA or hybrid state estimates. Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.

Hardware that fits

Apple entries show installed unified memory separately from a model budget based on an 8 GB reserve. That budget is a planning heuristic, not a fixed hardware limit.

No single GPU in our catalog has enough memory. Multi-GPU or CPU offload required.

Same 428B @ Q4_K_M, 1,048,576 context scenario, pre-loaded. Change the quant, shrink the context, or try a different preset.
Edit in the full tool

MiniMax-M3 428B at Q4_K_M with native 1M context needs about 405 GB of VRAM with all experts resident. MiniMax's third generation activates 23B parameters per token from a 428B total pool. The architecture uses block-sparse MSA for attention compute, but stores the key-value cache at full resolution. Those 23B active params don't shrink what has to stay loaded, because different tokens pick different experts.

How this is calculated

The 428B parameter pool requires 240 GB of weights at Q4_K_M. The 60-layer architecture consists of 3 dense layers plus 57 MoE layers, using group-query attention with 4 KV heads at head_dim 128.

Block-sparse MSA is an attention compute optimization that attends to the top-16 blocks of 128 tokens, not a storage compression, so the key-value cache is stored full-resolution. At the full 1M context the FP16 key-value cache uses 129 GB. Activation and software overhead adds about 37 GB.

Verdict

Cap the context before anything else. Block-sparse MSA speeds attention compute by attending only to top-16 128-token blocks, but the cache is stored at full sequence resolution so future queries can attend anywhere. At 256K the cache falls to about 32 GB, from 129 GB at 1M. Resident mode needs 6 cards of 80 GB or 3 H200 cards of 141 GB, with NVLink.

More MiniMax scenarios

Kimi K3.1 Pro at Q4_K_M
A 2.78T expert pool with 24 latent attention layers and fixed state in the other 69.
View details
Mistral Large 4 at Q4_K_M
A 1T expert pool with 63 latent cache owners and 44 layers that also store an index key.
View details
Muse Spark Open at Q4_K_M
Only 16 of its 62 layers grow with context, and the other 46 hold a 2,048 token window.
View details

Related guides

MiniMax-M3.1-Flash VRAM
See how sparse attention in every layer sets the cache.
View guide
minimax m3 pricing
Per-token cost for the minimax m3 family this VRAM entry serves.
View guide
Prompt optimization
Cutting tokens is the cheapest way to cut cost.
View guide

Frequently asked questions

Does block-sparse MSA reduce the key-value cache?
No. Block-sparse MSA optimizes attention compute by attending only to the top-16 blocks of 128 tokens, which speeds up the attention pass. The key-value cache is still stored at full sequence resolution so future queries can attend to any past token. The cache size is the same as standard group-query attention.
How does MiniMax-M3 compare to M2.7?
M3 has a 428B total pool with 23B active across 60 layers versus M2.7's 230B pool across 64 layers. M3 upgrades to a native 1M context window and block-sparse MSA compute. The M2.7 cache at 256K context was smaller. The M3 cache at 1M context is about 129 GB due to the longer window, not the architecture.
What is block-sparse MSA?
A sparse attention variant that computes attention only over selected blocks of the sequence instead of every token pair, which cuts attention compute as context grows. It doesn't shrink the key-value cache, which stays at full resolution. It trades a bit of fine-grained attention for much cheaper compute at the context lengths agentic workloads run.

Related tools

RAM Latency Calculator

Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.

Use tool

Power Cost Estimator

Estimate annual electricity costs for your PC, Server, or TV.

Use tool

Data Transfer Calculator

Estimate transfer times for files over USB, WiFi, Ethernet, and more.

Use tool

Memory and Storage Throughput Visualizer

Visualize the large speed gap between CPU cache, RAM, and storage.

Use tool