What happens when a GPU writes memory

summary

Writing data to global memory on an RTX 4090 is a complex, asynchronous process that moves data through a series of caches before it eventually reaches DRAM. Unlike reads, which stall the warp until data returns, stores are 'fire-and-forget' operations that rely on a write-through L1 and a sophisticated L2 replacement policy to manage data durability. The system ensures efficiency by coalescing memory accesses and using a 'dirty line' management strategy to prevent DRAM traffic bursts.

key points

1. The store process begins with the coalescer combining 32 individual lane accesses into the fewest possible 32-byte sectors, which then pass through a write-through L1 cache directly to the L2 cache.
2. L2 cache slices manage data using a re-reference prediction value (RRPV) and a specific '8-dirty rule' that proactively cleans dirty lines to ensure a steady flow of traffic to DRAM and avoid memory controller bottlenecks.
3. Memory visibility is managed through fences (membar), which force warps to wait until stores are visible at different scopes, ranging from the local SM's L1 to the entire system including the host CPU.

who should read it

GPU architects, CUDA developers, and systems engineers who want to understand the low-level hardware behavior of memory writes to optimize kernel performance.

discussion positive