Reproduce “Correct but Slow: An Empirical Study of the GPU Kernel Evaluation Gap in Modern Domain-Specific Languages” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tingxi Li, Ravishka Rathnasuriya, Wei Yang
Researchers at The University of Texas at Dallas conducted an empirical study revealing a substantial performance gap between GPU kernels implemented in modern Domain-Specific Languages (DSLs) and highly optimized vendor libraries. They found that current benchmarks admit functionally correct but orders of magnitude slower DSL kernels, and introduced lightweight heuristics to effectively flag and guide the repair of these inefficient implementations.
Key insights
- Current correctness-only benchmarks admit functionally valid DSL kernels that can be hundreds to over a thousand times slower than their optimized counterparts, failing to differentiate efficient from performance-poor implementations.
- Performance disparities in DSL kernels stem from identifiable root causes, including suboptimal user-space authoring choices (e.g., serial loops for reductions), compiler code-generation limitations (e.g., absent vectorization, sub-optimal auto-tuning), and library maturity gaps (e.g., lack of L2 cache awareness or algorithmic diversity).
- Combining a baseline-relative comparability screen with a baseline-independent roofline anchor can effectively identify inefficient kernels across diverse operators and guide targeted performance improvements without requiring comprehensive, labor-intensive benchmarking.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.