How much VRAM does GLM-5.1 754B (MoE) need at Q4_K_M? Zhipu 200K frontier

How this is calculated

Zhipu built GLM-5.1 with 64 layers, hidden size 8192, and 16 key-value heads. The 754B parameter pool at Q4_K_M requires 422 GB of weights regardless of routing. At the maximum native context window of 200K tokens, the FP16 key-value cache uses a significant 105 GB of memory. Standard activation and software overhead scale with model size, adding about 53 GB. If you offload cold experts and only keep the active 40B weights in VRAM, weight memory shrinks to 22.4 GB, which drops total resident usage to 141 GB. You'll need high-bandwidth hardware to run this MoE at acceptable speeds.

Verdict

The 580 GB resident total breaks into 422 GB of Q4 weights, 105 GB of FP16 KV cache at the 200K native context (16 KV heads at head_dim 128 across 64 layers, no MLA compression), and 53 GB of overhead. That's eight 80 GB cards with NVLink, or four 141 GB H200 cards with headroom. Active-only loading keeps only the routed 40B active weights in VRAM, shrinking the weight figure to 22.4 GB and the total to about 141 GB, but the 105 GB cache is bound by context length so it stays. So the real cost lever for active-only on this model is capping context: drop to 64K and the cache collapses to about 34 GB, dropping active-only to roughly 62 GB. The full 200K resident case is for serious infrastructure budgets and almost never beats the hosted API on cost.