Reproduce “ComFuse: Fusing Complex Memory-Intensive Subgraphs with Compute-Intensive Kernels For Modern GPU Architectures” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Di Mu, Tengyuan Jin, Zhenkun Wang, Jialin Yang, Yusen Li, Mian Huo, Shusong Guo, Gang Wang and 1 more
ComFuse introduces an automated compilation system to overcome performance bottlenecks in deep learning workloads by fusing complex memory-intensive subgraphs with compute-intensive kernels on modern GPU architectures. The system consistently outperforms TorchInductor by up to 1.97x and frequently matches or exceeds NVIDIA TensorRT by enabling on-chip data reuse and eliminating unnecessary global memory transfers.
Key insights
- Overlapping compute-intensive (MatMul) execution with memory-intensive post-processing (epilogue) can effectively hide latency and improve performance by exploiting complementary hardware resource demands.
- Complex memory-intensive subgraphs can be efficiently fused on-chip by decomposing them into stages with hierarchical reduction strategies, including intra-warp, intra-warp-group, and inter-CTA aggregation via DSMEM.
- Direct on-chip data transfer and consumption across multiple MatMuls (B2BGEMM) can eliminate redundant global memory access, but pipeline efficiency is sensitive to the computational balance between fused stages.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.