Kimi K3.1 Pro VRAM requirements at Q4 and 1M context

How this is calculated

The published config has 93 layers with hidden size 7,168. Of those, 69 are Kimi Delta Attention layers, which keep a fixed recurrent state (96 heads of width 128) no matter how long the prompt gets. That state adds up to 0.45 GB at batch one, and it grows with the number of concurrent sequences instead of with context. The other 24 layers are latent attention, and those are the only ones whose cache grows with every token. The expert layers send each token to 16 of 896 experts. New compared with Kimi K3, all 24 latent layers now use sparse attention that picks from a top 2,048 index and stores 128 wide index keys. Sparse selection cuts the attention work, not the stored history, because a later token can still ask for any earlier one. There's no public modeling file for K3.1 Pro, so this page keeps the same 1,024 values per latent layer as the Kimi K3 preset. The config implies 704 values (512 latent, 64 rope and 128 index), so the budget has headroom over that reading. At 1,048,576 tokens the cache is 51.5 GB. At 32K it shrinks to 1.61 GB and the total to 1715 GB, a saving of 55 GB. The 69 fixed state layers are the reason context stays cheap here, and the weights are the reason it doesn't matter much. For comparison, the Kimi K3 preset comes to 1781 GB in the same scenario. K3.1 Pro lands just under it, so a hardware plan for one carries over to the other. The estimate leaves out the vision encoder for image and video input, your engine's prefill workspace, and any speculative decoding buffers.

Verdict

Treat this as a multi-node model. Eight cards of 80 GB give 640 GB in total, which is well short of this estimate. The page's total takes 23 cards of 80 GB or 10 cards of 192 GB before sharding overhead. Two nodes of eight 192 GB cards offer 3072 GB, which clears the full context estimate on paper. Lowering context won't change that plan, since even 32K still needs about 1715 GB. Real savings have to come from the weights, so check which lower precision checkpoints your engine supports. The model ships under a custom license, so read it before commercial use. Measure resident weights, cache and peak prompt memory on the exact checkpoint, and check each card's allocation, because an aggregate fit can still fail on the busiest shard.