WGMMA is a Hopper-era GPU matrix multiply primitive. The interesting part is not only the instruction itself, but the execution style around it: work is arranged so that memory movement, synchronization, and matrix math overlap instead of forming a simple serial pipeline.
Working Model
At a high level:
- TMA or cooperative loads move tiles into shared memory.
- Producer and consumer stages coordinate through barriers.
- WGMMA consumes shared-memory tiles and accumulates into registers.
- The kernel tries to keep data movement and math in flight at the same time.
Why It Matters
Efficient GPU kernels are often limited by orchestration rather than raw FLOPs. If the kernel waits for every memory movement before issuing math, most of the machine sits idle. The async pattern makes the kernel look more like a small dataflow system.
Open Questions
- How many pipeline stages are enough before occupancy or shared memory becomes the bottleneck?
- Where is the clean boundary between compiler scheduling and manual kernel scheduling?
- How should this be explained from first principles without hiding behind CUDA jargon?