Which LLMs Fit in 48 GB of VRAM?

At Q4_K_M, the step up from 32 GB adds little. The largest model that fits is Mixtral 8x7B (MoE), at 32.8 GB, and the next size class up, the 70B models, sits just past the limit.

The gain is in the other columns. 54 of the 55 models that fit at Q4_K_M with 8K also fit at Q8_0, so the 30B class runs close to full quality. At Q8_0 with 8K, K2 Horizon 32B needs 43.0 GB, and Flint 40B (MoE) is the largest at 46.8 GB.

Llama 3.3 70B needs 50.0 GB at Q4_K_M with 8K, 2.0 GB over, and Qwen 2.5 72B needs 51.3 GB. For Llama, 42.7 GB of the total is weights and 2.7 GB is KV cache. The 10% allowance does the rest.

Some setups get a 70B model running at Q4_K_M on 48 GB anyway, with a short context and a runner that needs little extra. It's a squeeze with no room to spare. Q3_K_M is the safer route: Llama 3.3 70B needs 40.7 GB there and fits with room.

At Q4_K_M, 52 models still fit at 32K. Gemma 2 27B, Gemma 2 9B and Phi-4 14B drop out because they only go up to 8K or 16K tokens, not because they're too big.

Even at Q8_0 with 32K, Nanbeige4.3 33B fits at 46.3 GB, though it's tight. For 64K or 128K, open the model finder from the card below the table and change the context there.

Nvidia's RTX PRO 5000 Blackwell comes with 48 GB, and so do the older RTX 6000 Ada and RTX A6000. In data centers, the L40S has 48 GB too.

Two 24 GB cards, like a pair of RTX 3090s, add up to 48 GB. llama.cpp and vLLM can split a model across them, but each card needs some memory for its own buffers, and the model finder doesn't count that. Treat this list as a best case for two cards, and be careful with the tight rows.