Reproduce “Analyzing Reverse Address Translation Overheads in Multi-GPU Scale-Up Pods” on OpenResearch

About this paper

Amel Fatima, Tuan Ta, Bradford M. Beckmann

This study systematically characterizes Reverse Address Translation (NPA-to-SPA) overheads in multi-GPU scale-up pods, revealing performance degradation up to 1.4x for small, latency-sensitive collectives due to cold TLB misses. The research also shows that streaming memory access patterns limit the benefits of large L2-TLB capacities, informing architectural design for future interconnects.

Key insights

• Cold TLB misses are a primary bottleneck, causing significant performance degradation (up to 1.4x) for small, latency-sensitive collectives, which are common in ML inference workloads.
• The memory access patterns of ML collective operations are largely streaming with minimal temporal locality for translation entries, meaning that large L2-TLB capacities offer diminishing returns.
• Reverse Address Translation can account for a substantial portion, up to 30%, of the total round-trip latency for inter-node requests, especially in scenarios with frequent cold misses.

Summary by alphaXiv.