Ling-3.1-flash VRAM requirements at Q4 and 1M context

How this is calculated

The cache is small because of how the layers are built. Every eighth layer, nine in all, is latent attention that stores 576 values per token (a 512 value latent plus 64 rope values). The other 63 layers are Kimi Delta Attention, which keeps a fixed state instead of a cache that grows. Together that state is 0.28 GB at batch one, and it doesn't change with context length. In FP16 the latent layers cost 10,368 bytes per token, which makes 10.9 GB at 1M tokens, or 3 percent of the total. The runtime allowance alone, at 32.7 GB, is bigger than the whole cache. Two context numbers are in circulation for this model. The model card lists 1M tokens, while max_position_embeddings in the config is 262,144. This page uses 1M because that's what the card says, and it also shows 256K, where the total is 348 GB. Stopping at 256K trims 9.0 GB from the estimate, and 32K trims 11.6 GB for a total of 346 GB. Check which limit your server enforces before you size anything around the longer one.

Verdict

Budget for the weights first and treat the cache as a rounding error. By capacity alone the total needs 5 cards of 80 GB or 2 cards of 192 GB. Eight 80 GB cards give 640 GB, which covers this estimate on paper. Paper capacity isn't a deployment plan. The engine decides how experts and layers are split, which splits it supports, and how much workspace each card keeps for itself. The model card recommends SGLang, and vLLM also works. Speculative decoding needs extra memory that this estimate leaves out, and so do prefill workspace and communication buffers. Measure resident memory at your real context length and batch size before you order hardware.