Reproduce “Why Speculative Decoding Works Better Than Predicted on Sparse MoE Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ekagra Ranjan, Komal K. Teru, Bharat Venkitesh, Acyr Locatelli
The authors examine speculative decoding verification costs in recent sparse mixture-of-experts models at low batch sizes. They compare measured target efficiency with predictions based on activated-expert bytes, analyze expert reuse across adjacent tokens, and profile GPU kernels. For Qwen3-30B, measured target efficiency is 0.860 versus 0.511 predicted; these findings do not establish universal speedups across hardware.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.