Reproduce “Sawtooth Wavefront Reordering: Enhanced CuTile FlashAttention on NVIDIA GB10” on OpenResearch

About this paper

Yifan Zhu, Yekai Pan, Chen Ding

The authors reduce L2 miss traffic in tiled attention kernels by alternating the KV scan direction for successive query tiles. In CUDA and CuTile tests on NVIDIA’s GB10, this ordering lowers cache misses and raises kernel throughput. The non-causal tile-based CuTile result improves about 13%, while the causal comparison improves about 60%; these are tested-kernel measurements, not general guarantees.

Key insights

• The L1 cache functions primarily as a pass-through buffer for streaming FlashAttention data on the GB10, indicating that L2 cache behavior is the dominant factor in memory performance.
• L2 non-compulsory misses become prominent when the Key-Value (KV) matrix size approaches the L2 cache capacity, specifically around an 80K sequence length on the GB10 (20 MiB KV data vs. 24 MiB L2 cache).
• On the GB10, Cooperative Thread Arrays (CTAs) exhibit wavefront-like data reuse, but standard cyclic access patterns create reuse distances that exceed L2 capacity, leading to non-compulsory misses.

Summary by alphaXiv.