Reproduce “Arbitrage: Efficient Reasoning via Advantage-Aware Speculation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Monishwaran Maheswaran, Rishabh Tiwari, Yuezhou Hu, Kerem Dilmen, Coleman Hooper, Haocheng Xi, Nicholas Lee, Mehrdad Farajtabar and 3 more
Arbitrage routes draft reasoning steps by predicting whether the target will improve each step’s process-reward-model score, then keeps the draft or invokes the target. It compares expected improvement rather than applying RSD’s fixed quality cutoff. Across six settings, its router exceeds RSD at nearly all acceptance rates. On OlympiadBench, it reports up to 1.97× wall-clock speedup at matched accuracy, not token savings.
Key insights
- The primary inefficiency in prior step-level speculative decoding is its 'advantage-blind' nature, which decides to regenerate a step based on an absolute quality threshold rather than the *relative improvement* expected from the target model.
- An optimal routing policy can be framed as a thresholding decision on the 'step-level advantage' (the difference in quality between target and draft steps).
- Effective training of the ARBITRAGE ROUTER benefits from explicit step annotations (historical routing context) and class-balanced downsampling of training data to prevent bias towards over-accepting draft steps.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.