Reading the B in a model name
The B stands for billions of parameters, the learned numbers (weights) inside the network. Llama 3.1 comes in 8B, 70B and 405B sizes. Every one of those weights has to sit in memory before the model can run, which is why the count is the first number people check. It's usually a rounded label, not an exact figure, and for mixture of experts models one name can carry two numbers.
From parameters to gigabytes
Memory for the weights is parameters times bytes per parameter. At FP16 that's 2 bytes each, and at Q4_K_M it's about 0.61. For the three Llama 3.1 sizes:
- 8B: 16 GB at FP16, 4.9 GB at Q4_K_M
- 70B: 140 GB at FP16, 43 GB at Q4_K_M
- 405B: 810 GB at FP16, 248 GB at Q4_K_M
Eight 80 GB cards pool 640 GB, and the 405B model's FP16 weights alone don't fit in that. The LLM VRAM Calculator takes the full architecture, so you can check any model instead of guessing from its size.
What the count doesn't tell you
- Quality. Training matters as much as size. DeepMind's Chinchilla work found that a 70B model trained on more data beat the 280B Gopher, and that model size and training tokens should grow together.
- Compute per token. In a mixture of experts model only some of the parameters run for each token, so the active count sets the speed while the total sets the memory.
- Context cost. The KV cache depends on layers, KV heads and head size, not on the parameter count. In the calculator's model list, Phi-3 Mini (3.8B) has 32 KV heads against 8 for Llama 3.1 8B, so at 128K context its cache takes 52 GB against 18 GB, even with the smaller parameter count.