Flint 40B VRAM requirements at Q4 and 256K context

How this is calculated

Flint has 49 layers with hidden size 2048 and 4 KV heads at head dimension 128. Of those layers, 14 use full attention and 35 use a 513 token window, so most of the cache is a small fixed buffer. The 14 full layers cost 28,812 bytes per token in FP16. At 262,144 tokens the cache is 7.55 GB, at 128K it's 3.79 GB, at 32K it's 0.98 GB and at 8K it's 0.27 GB. The model routes each token to 6 of 256 routed experts and also runs a shared expert, but all of them count toward the weights. The checkpoint on Hugging Face is about 80 GB to download, so a Q4 conversion is a separate file and you'll need to check which formats your engine can load. The totals are 32.8 GB at 256K, 28.7 GB at 128K, 25.6 GB at 32K and 24.8 GB at 8K. The weights alone are over 20 GB, so context only decides the last few gigabytes and the card size you need is set mostly before the first prompt. That makes the line between card sizes very sharp. Even an 8K context doesn't fit a 24 GB card in this estimate, so that card would need a lower precision than Q4_K_M or some offload to system memory. A 32 GB card falls just short at the full context. It does hold 32K tokens, with a little room left. A Q8 cache narrows the gap, since it brings the full context total to 28.7 GB. On paper that fits a 32 GB card, though your engine has to support a Q8 cache for this architecture. The estimate leaves out prompt processing buffers, engine allocations and anything the display or other programs hold on the same card.

Verdict

Flint is the kind of model you can try on one workstation card, and the card size decides how much context you get. For a 32 GB card, plan on the short context end and measure how far you can push it. For two cards or one card in the 48 GB class, the full 256K scenario at 33 GB is within reach, and a single 48 GB card holds it on paper. Only about 2.5B parameters are active per token, so a setup that keeps expert weights in system memory is worth a test if your engine supports it. That trades memory for speed in ways this page can't predict, so try it with your own prompts. A machine with 64 GB of unified memory clears the full context estimate on paper, but check how much of that memory the operating system lets a single process use.