How much VRAM does Qwen 3.8 27B need at Q4_K_M? Dense 1M context mid-size
By TechCompare · Updated · How we check
Qwen 3.8 27B at Q4 needs ~29 GB at 128K context or ~95 GB at the full 1M window with YaRN. The KV cache, not the 17 GB of weights, sets the hardware. A 48 GB card handles it at 128K, and that's how most people should run it.
Calculator
Estimated VRAM required
94.2 GB
27.5B params at Q4_K_M, 1,048,576 token context, batch 1, inference.
Estimate accuracy: Weights within ~3% of most GGUF files from Q4_K_M up, ~7% for Q3_K_M and Q2_K. KV cache within ~5% for standard GQA models, ~10% for MLA or hybrid state estimates. Real VRAM may vary with framework (vLLM vs llama.cpp vs Transformers), Flash Attention, and driver overhead.
KV cache exceeds model weights: Consider lowering the context length to save on VRAM. Contexts between 8K and 64K are generally more typical for local setups.
Hybrid linear attention applied: 48 of 64 layers use fixed recurrent state, so only 16 full-attention layers contribute to the context-growing KV cache. The fixed state adds about 0.16 GB at batch 1 and does not grow with context length.
Hardware that fits
Unified memory machines show a model budget, not their installed memory. We hold back 8 GB for the system, or use the vendor's GPU limit when that's lower. It's a planning estimate, not a hard limit.
Multi-GPU setups
Pooled cards show up once a model outgrows the biggest single card of that kind. Each card past the first keeps about 1 GB for its own buffers, and that's counted here. Your inference engine has to support the split, and speed also depends on how the cards connect.
Qwen 3.8 27B at Q4_K_M needs about 95 GB of VRAM if you stretch it to a full 1M context with YaRN, past its 262,144 token native window, but the weights are only 17 GB and most of the rest is key-value cache. Alibaba's dense 27B is the practical local-deployable member of the Qwen 3.8 family.
How this is calculated
Weights at Q4_K_M are 16.8 GB. Only 16 of the 64 layers use full attention and keep a KV cache, with 4 KV heads at head_dim 256. That cache is 69 GB at the full 1M context and 8.6 GB at 128K, and overhead adds 9 GB at the long window. The other 48 are Gated DeltaNet layers that keep a fixed 0.16 GB state.
At a 1M window, most of this mid-size dense model's VRAM goes to the KV cache. At a 128K working context the total is about 29 GB and the model fits on a 48 GB card.
Verdict
Run it at 128K, where the total is 29 GB (16.8 GB of Q4 weights, 8.6 GB of FP16 KV cache and 2.7 GB of overhead). That fits a 48 GB RTX 6000 Ada or two 24 GB consumer cards. The full 1M window takes the cache to 69 GB and the total to 95 GB, which needs datacenter GPUs.
Most workloads stay under 128K, and RAG often works better than raw long context for retrieval tasks.
More Qwen scenarios
Related guides
Frequently asked questions
Why does a 27B model need 95 GB at 1M context?
Qwen 3.8 27B or Qwen3.5 27B?
Related tools
RAM Latency Calculator
Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.
Use toolPower Cost Estimator
Estimate annual electricity costs for your PC, Server, or TV.
Use toolData Transfer Calculator
Calculate file transfer time or the speed you need over USB, Wi-Fi or Ethernet, with latency.
Use toolMemory and Storage Throughput Visualizer
Visualize the large speed gap between CPU cache, RAM, and storage.
Use tool