Reproduce “REAL Loss: Optimizing Real Expected Acceptance Length for Speculative Decoding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yu Tian
The authors introduce REAL, a training loss for speculative decoding that weights proposed-token continuations by the branches surviving verification and exactly marginalizes each prefix’s final token. This addresses a mismatch in WTV’s product of average acceptance rates. In controlled experiments, REAL reaches mean probabilistic serving acceptance length 4.792 versus 4.749 for DBLast, on one target, drafter, and data recipe.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.