How much VRAM does Qwen 3.8 2.4T Max need at Q4_K_M? Alibaba's biggest 1M MoE

By TechCompare · Updated · How we check

Qwen 3.8 2.4T Max is a ~1720 GB resident deployment at 1M context. The license is worth reading too: Qwen 3.8 Max allows commercial use, but large model as a service businesses need a separate license, so check usage rights before sizing hardware.

Calculator

Estimated VRAM required

1,720 GB

2400B params at Q4_K_M, 1,048,576 token context, batch 1, inference.

Weights
1,464 GB
KV cache
98.8 GB
Overhead
157 GB

Estimate accuracy: Weights within ~3% of most GGUF files from Q4_K_M up, ~7% for Q3_K_M and Q2_K. KV cache within ~5% for standard GQA models, ~10% for MLA or hybrid state estimates. Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.

Hybrid linear attention applied: 69 of 92 layers use fixed recurrent state, so only 23 full-attention layers contribute to the context-growing KV cache. The fixed state adds about 0.60 GB at batch 1 and does not grow with context length.

Hardware that fits

Unified memory machines show a model budget, not their installed memory. We hold back 8 GB for the system, or use the vendor's GPU limit when that's lower. It's a planning estimate, not a hard limit.

No single card or machine in our catalog has enough memory. The multi-GPU setups below do.

Multi-GPU setups

8× NVIDIA B300
Datacenter · 216 GB per card
2,304 GB
75% used

Pooled cards show up once a model outgrows the biggest single card of that kind. Each card past the first keeps about 1 GB for its own buffers, and that's counted here. Your inference engine has to support the split, and speed also depends on how the cards connect.

Qwen 3.8 2.4T Max at Q4_K_M with a 1M context needs about 1720 GB of VRAM with every expert resident. Alibaba's biggest Qwen 3.8 model activates 95B parameters per token from the 2.4T pool. That doesn't shrink what has to stay loaded, because different tokens pick different experts. The Qwen3.8-Max license allows commercial use, with conditions at large scale, so read it before you deploy.

How this is calculated

The 2.4T parameter pool at Q4_K_M is 1464 GB of weights. Only 23 of the 92 layers use full attention and keep a key-value cache, with 4 KV heads at head_dim 256. That cache is 95 GB at the full 1M context, which is past the 262,144 token native window, so it relies on the extension Qwen documents. The other 69 are Gated DeltaNet layers with a fixed 0.60 GB state, and overhead adds about 157 GB. The Qwen3.8-Max license allows commercial use. A model as a service or AI work assistant business with more than US$50 million in revenue over 12 months needs a separate license from Qwen.

Verdict

Check the license before sizing hardware. Qwen 3.8 Max allows commercial use, but a model as a service business above US$50 million in revenue over 12 months needs a separate license from Qwen. If you self-host, resident mode needs 22 cards of 80 GB or 13 H200 cards of 141 GB.

More Qwen scenarios

Kimi K3.1 Pro at Q4_K_M
A 2.78T expert pool with 24 latent attention layers and fixed state in the other 69.
View details
Mistral Large 4 at Q4_K_M
A 1T expert pool with 63 latent cache owners and 44 layers that also store an index key.
View details
Muse Spark Open at Q4_K_M
Only 16 of its 62 layers grow with context, and the other 46 hold a 2,048 token window.
View details

Frequently asked questions

Is Qwen 3.8 Max open for commercial use?
Yes, with conditions. The Qwen3.8-Max license lets you use, modify, sell and deploy the model. A product with more than 100 million monthly active users or US$20 million in monthly revenue must show the model name in its interface. A model as a service or AI work assistant business with more than US$50 million in revenue over 12 months needs a separate license from Qwen. Read the full text before you ship.
Should I run Qwen 3.8 2.4T Max or Qwen3.5 122B locally?
Qwen3.5 122B for almost everyone. It fits on a single H200 of 141 GB resident and has clean Apache-2.0 licensing. The 2.4T Max needs 13 H200 cards of 141 GB resident and carries license conditions to read before you ship to customers.