Which LLMs Fit in 32 GB of VRAM?
By TechCompare · Updated · How we check
54 of the 108 models in the calculator fit in 32 GB at Q4_K_M with 8K context. Nanbeige4.3 33B ranks first at 24.2 GB, and Flint 40B (MoE) is the largest that fits, at 27.1 GB.
32 GB is the memory on the RTX 5090. Next to 24 GB, it doesn't open up a new size class, but it gives the 30B class room to breathe: longer context, and a few models that miss on 24 GB. This page checks every model in the LLM VRAM Calculator against 32 GB at Q4_K_M and Q8_0, with 8K and 32K tokens of context.
Every total counts the weights, the KV cache and a 10% allowance for the runtime. Pick a number to open the calculator with that model and setup.
Models that fit in 32 GB of VRAM
Best pick first, by size and release date. Each cell is the total memory the model needs. Tight means under 10% of the memory is left. Pick a number to open it in the calculator.
Just too big at Q4_K_M with 8K context
Totals include the weights, the KV cache at FP16 with a batch size of 1, and a 10% allowance for the runtime.
What fits at Q4_K_M
Nanbeige4.3 33B ranks first at 24.2 GB, followed by K2 Horizon 32B at 25.7 GB. Both miss on a 24 GB card, so they're the clearest gain from the extra memory. The largest model that fits is Flint 40B (MoE), at 27.1 GB. It runs about 2.5B parameters per token, so it's quick for its size.
The order comes from the model finder's best pick, which weighs size against release date. Treat it as a short list, not a verdict.
32B models at 32K
On a 24 GB card, the dense 32B models drop out at 32K. Here most of them fit. Qwen3 32B needs 31.0 GB at 32K, which is tight but fits, and DeepSeek R1 Distill 32B needs 31.3 GB. K2 Horizon 32B has more parameters than its name suggests and needs 32.8 GB, so it misses.
Tight means little memory is left over. A chat that grows past 32K, or a second app on the GPU, can push a tight model over.
Q8_0 on 32 GB
Q8_0 tops out around 24B to 26B. Gemma 4 26B (MoE A4B) needs 29.8 GB and Mistral Small 3 24B needs 29.5 GB, both with 8K. Qwen 3.8 27B misses at 32.9 GB.
Gemma 4 26B still fits at Q8_0 with 32K, at 30.4 GB, thanks to its sliding window layers. In all, 36 models fit at Q8_0 with 8K, against 54 at Q4_K_M.
What 32 GB can't reach
Mixtral 8x7B (MoE) needs 32.8 GB and misses by 0.8 GB. It fits at Q3_K_M, at 26.6 GB, but it's from Dec 2023, and newer models on this list do more in the same space.
Past that, there's a big jump to the 70B class. Llama 3.3 70B needs 50.0 GB at Q4_K_M and still 40.7 GB at Q3_K_M. For those, see the 48 GB and 64 GB Mac lists.
Cards with 32 GB
The RTX 5090 is the main 32 GB card. AMD's Radeon AI PRO R9700 and Nvidia's RTX 5000 Ada also have 32 GB, for workstations. The RTX 5090 has far more memory bandwidth than either, so it runs the same models faster.
A Mac with 32 GB doesn't reach this list. macOS and your apps share its memory, so the calculator gives it 24 GB for models, which matches the 24 GB list.
Other memory sizes
Graphics cards
Macs and unified memory
Frequently asked questions
What's the best LLM for an RTX 5090?
Can the RTX 5090 run a 70B model?
Is 32 GB worth it over 24 GB for LLMs?
Can 32 GB of VRAM run Mixtral 8x7B?
Does the RTX 5090 fit Gemma 4 31B at 32K?
Related tools
LLM VRAM Calculator
Calculate the VRAM needed to run or fine-tune any LLM at any quantization.
Use toolLLM API Pricing Calculator
Compare API costs across major models (OpenAI, Anthropic, Google) with prompt caching.
Use toolLLM Token Counter
Count tokens in any prompt for GPT, Claude, Gemini, and Llama with exact OpenAI tokenization.
Use toolPower Cost Estimator
Estimate annual electricity costs for your PC, Server, or TV.
Use tool