TechCompare LogoTechCompare

CPU cache hierarchy explained: L1, L2, L3 and why they exist

The cache hierarchy is the most important architectural feature of a modern CPU that most developers never think about. Understanding it explains why array-of-structs vs struct-of-arrays matters, why linked lists are slow, and why optimizing for cache locality can speed up code by an order of magnitude.

CPU cores execute instructions in fractions of a nanosecond. RAM responds in 50-80 nanoseconds. Without cache, a CPU would spend 99% of its time waiting for data. The cache hierarchy (L1, L2, L3) bridges this gap by keeping frequently accessed data closer to the execution units. Each level is larger but slower than the one before it.

By TechCompare · Updated

Hardware tier
CPU Cache
On-die processor cache levels
Topic focus
CPU cache hierarchy
cache-hierarchy
Try the live Latency Visualizer. See how a file moves across CPU cache, RAM, and storage.
Edit in the full tool

How this is calculated

The hierarchy works on the principle of locality: if a program accesses a memory address, it's likely to access nearby addresses soon (spatial locality) and the same address again soon (temporal locality). L1 caches the most recently used data at the smallest granularity (64-byte cache lines). L2 catches L1 evictions. L3 catches L2 evictions and serves as a shared pool for inter-core communication. Cache design is a trade-off between latency (smaller is faster), hit rate (larger catches more), and power consumption (larger uses more). Modern CPUs spend roughly 30-40% of their die area on cache.

Verdict

Three tiers bridge a 50-80 ns gap between the core and RAM, and the whole stack runs on locality. L1 caches recent data in 64-byte lines, L2 catches L1 evictions per core, and L3 pools L2 evictions across cores for inter-core sharing. The design trades hit rate (larger catches more) against latency and power (larger costs both). Modern CPUs spend roughly 30-40% of die area on cache, which is why data layout decisions like struct-of-arrays reshape performance.

More Latency scenarios

L1 vs L2 cache
L1 cache is the fastest memory in a computer, typically 1 ns latency (3-5 CPU cycles) and 32-64 KB per core.
View details ➜
L3 cache vs RAM
L3 cache (also called Last Level Cache or LLC) is shared across all cores in a CPU chiplet, typically 16-96 MB, with latency of 10-15 ns.
View details ➜
DDR4 vs DDR5 latency
DDR5 roughly doubles the peak bandwidth of DDR4 (from ~50 GB/s to ~100 GB/s per module), but true latency measured in nanoseconds is nearly unchanged.
View details ➜

Related guides

L1 vs L2 cache
The first cache tier in this hierarchy.
View guide ➜
L3 cache vs RAM
The L3-to-RAM cliff at the bottom of the hierarchy.
View guide ➜

Frequently asked questions

Why do CPUs have three levels of cache?
Because fast memory is small and cheap memory is slow, and the hierarchy fakes having both. L1 catches the hottest 64-byte lines at 1 ns, L2 catches L1's evictions at 3-4 ns, and L3 pools L2's evictions across cores at 10-15 ns. Without the stack, every RAM access at 50-80 ns would leave the CPU idle most of the time.
What is a cache line?
The unit memory moves in: 64 bytes at a time on modern x86 and ARM. Reading one byte pulls its whole 64-byte line into L1, which is why sequential access is nearly free (the next bytes are already there) and skipping across memory wastes most of every fetch. Struct-of-arrays layouts exist to keep hot fields packed into as few lines as possible.
Why are linked lists slow on modern CPUs?
Pointer chasing defeats the cache. Each node lives wherever the allocator put it, so every step is a fresh miss that walks the hierarchy or hits RAM. An array of the same items is contiguous, prefetches cleanly, and can outperform the linked list by an order of magnitude even when the list's algorithmic complexity looks better on paper.

Related tools

RAM Latency Calculator

Convert DDR3/DDR4/DDR5 timings (CL, tRCD, tRP, tRAS) into true latency in nanoseconds.

Use tool ➜

RAID Calculator

Calculate usable capacity and fault tolerance for RAID 0, 1, 5, 6, and 10.

Use tool ➜

Display Bandwidth Calculator

Check if your HDMI/DP cable supports your resolution and refresh rate.

Use tool ➜