Which LLMs Fit in 16 GB of VRAM?

Q4_K_M is the calculator's default quant and a common pick for running models at home. It stores 0.61 bytes per parameter on average, so the weights take about 0.61 GB for every billion parameters.

At 8K context, that leaves room for dense models up to Qwen3 14B, which needs 11.4 GB. The largest model that fits is gpt-oss 20B, a mixture of experts model with 20B parameters. It needs 13.7 GB.

The table ranks models the way the model finder's best pick does. Size counts, and so does age: a model counts as half its size for every 12 months since its release, since newer models tend to do more with fewer parameters. A mixture of experts model counts between its active and total size. That's why Gemma 4 12B, from May 2026, ranks above Qwen3 14B from Apr 2025. The order is a starting point. Try the top few on your own tasks before you settle on one.

The KV cache stores keys and values for every token in the context, so it grows as the context gets longer. How fast it grows depends on how the model's attention works, which is why models of the same size can land far apart at 32K.

• Qwen3 14B needs 11.4 GB at 8K and 15.9 GB at 32K. It still fits, but it's tight, with little left over on the card.
• Phi-3 Mini 3.8B is small, but every one of its attention heads keeps its own keys and values. Its KV cache takes 12.9 GB at 32K, and the model no longer fits.
• Gemma 4 12B uses sliding window attention on most of its layers, and those layers only cache the most recent tokens. Its KV cache at 32K is 0.9 GB, against 5.4 GB for Qwen3 14B.

For long chats or big documents, read the 32K columns first. A model that's tight at 32K can run out of room if you push past that.

Q8_0 stores 1.06 bytes per parameter, against 0.61 for Q4_K_M. A bigger quant keeps more detail in each weight, so it's worth using when the model fits.

At 8K context, Gemma 4 12B still fits at Q8_0 with 14.6 GB, though it's tight. Qwen3 14B needs 18.7 GB and is 2.7 GB over.

At 32K, most 8B and 9B models still fit, but not all. Llama 3.1 8B needs 14.1 GB and Qwen3.5 9B needs 12.5 GB, while Granite Switch 4.1 8B needs 16.2 GB and misses. In all, 25 models fit at Q8_0 with 32K, against 33 at Q4_K_M with 8K.

Nothing larger than gpt-oss 20B fits at Q4_K_M, even at 8K. The common 24B to 32B models are out of reach without help.

A mixture of experts model only runs a few of its experts for each token, but all of them have to sit in memory. LFM2.5 24B-A2B uses about 2B parameters per token and still needs 16.3 GB, just over the limit.

If a model misses by a little, you have two options. A smaller quant like Q3_K_M shrinks the weights, at some cost in quality. Runners like llama.cpp can also keep some layers in system RAM, which works but makes every token slower, since system RAM has far less bandwidth than VRAM.

Nvidia's RTX 5060 Ti 16GB, RTX 5070 Ti and RTX 5080 all have 16 GB, and so does AMD's RX 9070 XT. Nvidia also sells the RTX 5060 Ti with 8 GB, so check which one you have. The faster cards have more memory bandwidth, which speeds up each token but doesn't change what fits.

If the card also drives your monitors, the desktop and open apps use part of its memory. A model marked tight may not load then, so leave some room or close what you can.

A Mac with 16 GB is a different case. macOS and your apps share that memory with the model, so the calculator gives a 16 GB Mac 8 GB for models. Its list is on the 8 GB VRAM page.