How much VRAM does GLM-5.1 754B (MoE) need at Q4_K_M? Zhipu 200K frontier
Self-hosting GLM-5.1 in resident mode requires a substantial hardware setup. You'll need at least eight 80 GB datacenter cards with NVLink, or four 141 GB H200 cards. For typical applications, the official API is the most economical starting point.
GLM-5.1 754B at Q4_K_M with native 200K context needs about 580 GB of VRAM with all experts resident in GPU memory. That's the baseline you'll size production cluster deployments against. This giant Mixture of Experts from Zhipu activates 40B parameters per token from its 754B total parameter pool. That doesn't shrink what has to stay loaded, because different tokens pick different experts. Moving expert weights to system RAM cuts VRAM, but the host then has to hold them and every token gets slower.
By TechCompare · Updated
Calculator
Estimated VRAM required
580 GB
754B params at Q4_K_M, 200,000 token context, batch 1, inference.
Estimate accuracy: Weights within ~2%. KV cache within ~5% for standard GQA models, ~10% for MLA or hybrid state estimates. Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.
Hardware that fits
Apple entries show installed unified memory separately from a model budget based on an 8 GB reserve. That budget is a planning heuristic, not a fixed hardware limit.
No single GPU in our catalog has enough memory. Multi-GPU or CPU offload required.
How this is calculated
Zhipu built GLM-5.1 with 64 layers, hidden size 8192, and 16 key-value heads. The 754B parameter pool at Q4_K_M requires 422 GB of weights regardless of routing. At the maximum native context window of 200K tokens, the FP16 key-value cache uses a significant 105 GB of memory. Standard activation and software overhead scale with model size, adding about 53 GB. You'll need high-bandwidth hardware to run this MoE at acceptable speeds.
Verdict
The 580 GB resident total breaks into 422 GB of Q4 weights, 105 GB of FP16 KV cache at the 200K native context (16 KV heads at head_dim 128 across 64 layers, no MLA compression), and 53 GB of overhead. That's eight 80 GB cards with NVLink, or four 141 GB H200 cards with headroom. The 40B active count doesn't lower the resident total, because any token can route to any expert. Capping context is the lever you control: at 64K the cache falls to about 34 GB. The full 200K resident case is for serious infrastructure budgets and almost never beats the hosted API on cost.
More GLM scenarios
Related guides
Frequently asked questions
What hardware is best to run GLM-5.1 in full resident mode?
Can I run this model on consumer hardware?
Can I run GLM-5.1 754B on consumer hardware?
What does expert offload feel like in practice?
Related tools
RAM Latency Calculator
Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.
Use tool ➜Power Cost Estimator
Estimate annual electricity costs for your PC, Server, or TV.
Use tool ➜Data Transfer Calculator
Estimate transfer times for files over USB, WiFi, Ethernet, and more.
Use tool ➜Memory and Storage Throughput Visualizer
Visualize the massive speed difference between CPU cache, RAM, and storage.
Use tool ➜