TechCompare LogoTechCompare

How much VRAM does gpt-oss 20B need at Q4_K_M? OpenAI's first open-weights model

gpt-oss 20B at Q4_K_M is the model to download if you've never run a local LLM before in 2026. Apache 2.0, runs on anything, decent reasoning for the size, and OpenAI lineage. Not as strong as Qwen3 30B-A3B at the same memory footprint, but the brand recognition matters and the licensing is the cleanest in the open-weights ecosystem.

gpt-oss 20B at Q4_K_M with native 128K context needs about 19.4 GB of VRAM with all experts resident, dropping to roughly 9.3 GB with active-only weight loading. It's an MoE with 3.6B active parameters per token, 24 layers, an unusual hidden size of 2944, head_dim 64, and 128K native context. The 20B variant of OpenAI's first Apache 2.0 release is the smallest model in the family and is intentionally sized for laptops and consumer GPUs.

By TechCompare · Updated

Total VRAM required
19.4 GB
gpt-oss 20B (MoE) at Q4_K_M
Weights
11.2 GB
20B params
KV cache
6.4 GB
128K tokens, FP16 KV

Calculator

Estimated VRAM required

19.4 GB

20B params at Q4_K_M, 131,072 token context, batch 1, inference.

Weights
11.2 GB
KV cache
6.4 GB
Overhead
1.8 GB

Estimate accuracy: Weights within ~2%. KV cache within ~5% for standard GQA models, ~10% for MLA (DeepSeek). Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.

Hardware that fits

RTX 3090
Consumer
24 GB
81% used
RTX 5090
Consumer
32 GB
61% used
A100 40GB
Datacenter
40 GB
49% used
Apple M3 Max 64GB
Unified
48 GB
40% used
Same 20B @ Q4_K_M, 131,072 context scenario, pre-loaded. Change the quant, shrink the context, or try a different preset.
Edit in the full tool

How this is calculated

11.2 GB of weights at Q4_K_M plus a 6.4 GB KV cache (at the full 128K context window) and ~1.8 GB overhead. Active-only loading shrinks weights to 3.6B * 0.56 = 2.0 GB while keeping the same KV cache, totaling ~9.3 GB - it runs on a 12 GB GPU or a unified memory device.

Verdict

The total stacks to 19.4 GB resident at the full 128K context: 11.2 GB of Q4 weights plus 6.4 GB of FP16 KV cache and 1.8 GB of activation overhead. A 24 GB consumer card (RTX 3090/4090/5090) handles resident mode comfortably, while active-only loading shrinks weights to roughly 2 GB and drops the total to about 9.3 GB, which fits a 12 GB card or a unified-memory phone. Active-only trades speed for size, since cold experts fetch through system RAM at a per-token PCIe round-trip, typically landing at 15-30 tok/sec on consumer cards versus 60+ all-resident. Apache 2.0 plus OpenAI lineage is the differentiator against Qwen3 30B-A3B at the same footprint, which is otherwise the stronger-raw-quality alternative.

More gpt-oss scenarios

DeepSeek V4 Pro 1.6T (MoE) at Q4_K_M
DeepSeek V4 Pro 1.6T at Q4_K_M with the full 1M-token context needs about 1012 GB of VRAM with every expert resident - this is the real shape of the model and the number to plan a deployment against.
View details ➜
Llama 4 Scout (17B/109B) at Q4_K_M
Llama 4 Scout at Q4_K_M with its native 10M context needs about 2231 GB of VRAM with all 109B params resident - that's the number you size hardware against.
View details ➜
Qwen3.5 122B (MoE) at Q4_K_M
Qwen3.5 122B at Q4_K_M with native 256K context needs about 104 GB of VRAM with all 122B params resident.
View details ➜

Related guides

Prompt optimization
Cutting tokens is the cheapest way to cut cost.
View guide ➜

Frequently asked questions

What does '3.6B active' actually mean for inference speed?
Per token, the model only computes through 3.6B parameters worth of expert weights even though all 20B must be reachable. With active-only loading and CPU offload the cold experts add a per-token PCIe round-trip - typical numbers are 15-30 tokens/sec on consumer hardware vs 60+ all-resident.
Should I pick gpt-oss 20B or Qwen3 30B-A3B?
Qwen3 30B-A3B is generally stronger at the same active footprint (3B active vs 3.6B). gpt-oss has cleaner licensing and OpenAI lineage. Both fit on the same hardware - try whichever fits your stack first.
What GPU do I need to run gpt-oss 20B all-resident?
A 16 GB+ GPU for full resident mode at the 128K native context: 11.2 GB of weights plus 6.4 GB KV cache plus 1.8 GB overhead = roughly 19.4 GB. An RTX 4080 (16 GB) won't quite fit all-resident but a 24 GB RTX 3090/4090 handles it comfortably. For active-only loading (just 3.6B active params), a 12 GB card works.
Why does active-only loading change the VRAM math?
Because only the per-token active experts need to stay resident. With a 3.6B active subset out of a larger total pool, weight VRAM shrinks to a fraction of the full model, and the KV cache ends up the dominant term at long context. The trade is cold-expert fetch latency over PCIe, which caps throughput when requests touch many different experts in sequence.

Related tools

RAM Latency Calculator

Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.

Use tool ➜

Power Cost Estimator

Estimate annual electricity costs for your PC, Server, or TV.

Use tool ➜

Data Transfer Calculator

Estimate transfer times for files over USB, WiFi, Ethernet, and more.

Use tool ➜

Memory and Storage Latency Visualizer

Visualize the massive speed difference between CPU cache, RAM, and storage.

Use tool ➜