How much VRAM does Nemotron 3 Ultra 550B A55B (MoE) need at Q4_K_M? NVIDIA 1M Mamba-2

48 of 108 transformer blocks are Mamba-2 state-space layers. Mamba-2 maintains a fixed-size recurrent state instead of growing a key-value cache with context length. Only the 12 attention layers contribute to the cache. At 1M context the cache is about 13 GB, roughly an order of magnitude smaller than a pure-attention model of the same layer count.

Super is 120B total with 12B active. Only 8 of its 88 blocks are attention layers, at 2 KV heads, so its cache is 9 GB at 1M context. Ultra is 550B total with 55B active, 12 attention layers at 2 KV heads and 48 Mamba-2 layers, which gives a 13 GB cache at 1M context. Ultra's cache is a bit larger because it has more attention layers, but the weights drive the difference: Ultra has 4.6x the parameter count, so its resident total is 353 GB versus Super's 83 GB.

It means the reference VRAM numbers assume datacenter cards and many concurrent requests, not an interactive session on a workstation. A solo user on a single card gets the memory story but not the throughput story, which is where a shared deployment pays off.