Which LLMs Fit in 8 GB of VRAM?

Nearly every 7B to 9B model fits at Q4_K_M with 8K of context. Gemma 2 9B is the exception. The largest that fits is Qwen3.5 9B, at 6.9 GB. Granite Switch 4.1 8B ranks first at 7.4 GB. That's tight, so close other apps that use the GPU before you load it.

The ranking follows the model finder's best pick. It weighs size against release date, since a new 8B model often does better than an older one of the same size. Treat it as a short list and try the top few on your own prompts.

The KV cache grows with every token of context, and on 8 GB it often decides what fits. Llama 3.1 8B needs 6.6 GB at 8K and 10.1 GB at 32K, so it drops out. Qwen3.5 9B is a bigger model, but it still fits at 32K with 7.8 GB.

The difference is the attention. Qwen3.5 uses linear attention on three of every four layers, and those layers keep a fixed size state instead of a cache that grows. Its KV cache at 32K is 1.1 GB, against 4.3 GB for Llama 3.1 8B.

If you work with long documents on 8 GB, read the 32K columns first.

Q8_0 stores 1.06 bytes per parameter, against 0.61 for Q4_K_M. At Q8_0, no 8B model fits. Llama 3.1 8B needs 10.6 GB with 8K, and the largest model that fits is Gemma 4 E2B at 6.1 GB.

For most uses, an 8B model at Q4_K_M gives better answers than a 4B model at Q8_0. The extra parameters usually count for more than the extra bits. Q8_0 makes sense here for small models you already like, when you want them as close to the original as possible.

Gemma 2 9B needs 8.4 GB and Gemma 4 12B needs 8.6 GB at Q4_K_M with 8K, both just over. At Q3_K_M, Gemma 4 12B comes down to 7.0 GB and fits. A smaller quant costs some quality, but a newer 12B model at Q3_K_M can still beat an older 8B at Q4_K_M. Try both on your own tasks.

Runners like llama.cpp and Ollama can also keep some layers in system RAM. The model runs, but every token gets slower, since system RAM has far less bandwidth than the card.

Nvidia's RTX 5060 and RTX 4060 have 8 GB, as do the 8 GB versions of the RTX 5060 Ti and AMD's RX 9060 XT. So do most gaming laptops with an RTX 4060 or RTX 5060. If the same GPU drives your screens, the desktop takes a slice of that memory, and a model marked tight may not load.

A Mac with 16 GB lands on this list too. macOS and your apps share its memory with the model, so the calculator gives it 8 GB for models and keeps 8 GB back. That's a planning estimate, not a limit Apple sets.

If you're shopping, the 12 GB and 16 GB lists show what the next step up gets you.