GLM-5.3-Flash VRAM requirements: about 235 GB at Q4 and 1M context

By TechCompare · Updated · How we check

GLM-5.3-Flash needs about 235 GB in this Q4, 1M, batch-one budget. It includes the Transformers sparse-cache layout and fixed KDA state.

Calculator

Estimated VRAM required

235 GB

320B params at Q4_K_M, 1,048,576 token context, batch 1, inference.

Weights
196 GB
KV cache
17.8 GB
Overhead
21.5 GB

Estimate accuracy: Planning estimate. Rounded model counts, tensor formats, cache packing, and workspace can change actual allocation. Confirm the exact checkpoint and serving engine with a load test.

GLM-5.3-Flash uses 34 fixed-state KDA layers and 11 sparse MLA layers. This cache budget follows the Transformers implementation, including its unpooled indexer arrays. Optimized serving engines can use smaller caches. Q8 is a sizing scenario and requires backend support.

Hybrid linear attention applied: 34 of 45 layers use fixed recurrent state, so only 11 sparse-attention layers contribute to the context-growing KV cache. The fixed state adds about 0.15 GB at batch 1 and does not grow with context length.

Hardware that fits

Unified memory machines show a model budget, not their installed memory. We hold back 8 GB for the system, or use the vendor's GPU limit when that's lower. It's a planning estimate, not a hard limit.

Apple M5 Ultra 256GB
Unified
~248 GB
95% of estimate
NVIDIA B300
Datacenter
288 GB
82% used
Apple M5 Ultra 512GB
Unified
~504 GB
47% of estimate

Multi-GPU setups

4× RTX PRO 5000 72GB
Pro · 59.4 GB per card
288 GB
83% used

Pooled cards show up once a model outgrows the biggest single card of that kind. Each card past the first keeps about 1 GB for its own buffers, and that's counted here. Your inference engine has to support the split, and speed also depends on how the cards connect.

GLM-5.3-Flash combines a roughly 320B parameter pool with about 18B parameters active per token. Its attention layout matters as much as its MoE label: 34 layers keep fixed linear-attention state, while 11 sparse layers retain context-dependent storage. This page estimates 195.20 GB of Q4 weights and a total of 234.39 GB using FP16 cache and the Transformers implementation. Other serving engines can allocate the sparse indexer differently.

How this is calculated

The configuration specifies 45 layers, hidden size 4,096, 64 KDA heads of width 128, and a four-position convolution kernel. Each KDA layer retains a 64 x 128 x 128 FP32 recurrent state plus a BF16 convolution state for three projections. Across 34 layers that is about 0.150 GB at batch one, independent of context length.

The sparse layers cache a shared 512-value MLA latent rather than separate expanded query-head keys and values. The linked Transformers code also stores unpooled indexer keys, gate scores, and validity values, adding 128 + 128 + 1 values per token in each sparse layer.

Its total logical cache budget is therefore 11 x 769 values per input token, or 17.74 GB in FP16 at 1,048,576 tokens. Including 10% general overhead and fixed state gives 234.39 GB, about 218.29 GiB. At 32K context the total is 215.48 GB. An optimized pooled indexer can reduce this cache allocation.

Verdict

One 80 GB card isn't enough, even at 32K, where the Q4 scenario still takes 3 cards of 80 GB. Linear attention keeps most layers from accumulating a full sequence cache, but it doesn't make the 320B weight pool disappear.

Use the total as an implementation-specific planning budget, then compare it with your serving engine's measured allocation. Keep room for vision activations, prompt-processing workspace, and per-shard imbalance when evaluating a multi-GPU setup, and check that your backend offers a Q4 or Q8 cache.

More GLM scenarios

Kimi K3.1 Pro at Q4_K_M
A 2.78T expert pool with 24 latent attention layers and fixed state in the other 69.
View details
Mistral Large 4 at Q4_K_M
A 1T expert pool with 63 latent cache owners and 44 layers that also store an index key.
View details
Muse Spark Open at Q4_K_M
Only 16 of its 62 layers grow with context, and the other 46 hold a 2,048 token window.
View details

Frequently asked questions

Why doesn't the calculator use all 64 attention heads for KV?
The sparse path caches the compressed MLA latent and expands it for attention computation. Counting a separate full-width cache for every query head would overstate the retained storage. This preset uses one shared latent per token per sparse layer and separately includes indexer arrays. Query projection dimensions describe work performed by attention, not necessarily tensors that remain in the cache.
Does sparse attention store only the selected tokens?
Not in the implementation modeled here. Selecting a small set of positions reduces attention work, but the cache retains latent states for the sequence and the indexer keeps information used to choose those positions. The preset follows the linked Transformers cache writes. A runtime that pools indexer storage or uses a different packed layout can have a smaller cache, so a benchmark should identify its engine and revision.
How does batch size affect the linear-attention state?
Each concurrent sequence needs its own recurrent and convolution state. Doubling batch doubles that fixed state and the sequence cache, while resident weights are shared. The fixed state doesn't double when you double one sequence's context length. This distinction is why treating all 45 layers as ordinary full attention would give an incorrect memory curve for GLM Flash.
Are 320B and 18B exact checkpoint byte counts?
They're the publisher's rounded total and active parameter figures. The preset uses 320B as a weight planning input and doesn't infer tensor bytes from the 18B active count. Actual allocation depends on quantized tensor blocks, scales, modules that retain higher precision, and optional components. Inspect the checkpoint's tensor inventory and loaded memory to reconcile a deployment with this rounded estimate.
Will changing reasoning effort change resident model size?
Reasoning effort changes the generation workload rather than this architecture's stored weight pool. Longer outputs can grow context and occupy a request slot for longer, increasing cache usage and concurrency pressure. Size the service around combined prompt and output length, not only average input size. The publisher documents low, high, and max effort settings, so load tests should use the effort level you'll actually serve.