Reproduce “Understanding GPU Megakernels: Why Residency Can Mislead” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xinwei Qiang, Ziyi Xu, Chang Chen, Zaifeng Pan, Zhongkai Yu, Yue Guan, Yufei Ding

The authors compare separate kernels with PDL against persistent execution for batch-one LLM decoding. PDL lets a successor begin setup and independent data loads before its producer finishes, but waits before consuming dependent results. In a reconstruction of ThunderKittens’ Llama-3.2-1B decode, separate kernels reduce latency by 60.9 μs, or 5.7%, per step; this does not establish that separate kernels generally outperform megakernels.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.