Reproduce “SpecPilot: Autonomous Algorithm Discovery and Systems Optimization for Speculative Decoding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kunjun Li, Wenhao Chai, Yihao Liang, Yuezhou Hu, Shuyao Xu, Yi Pan, William X. Zheng, Kannan Ramchandran and 2 more
SpecPilot turns speculative-decoding optimization into a deployment-specific search: it measures bottlenecks, tests fixes with parallel agents, and validates performance and outputs before promotion. On one-request Qwen3.8-27B BF16, DFlash2, and RTX PRO 6000, it reaches 10.87× decoding output-rate speedup over autoregressive decoding, versus stock’s 4.46×. The result is deployment-specific; TLV is 13% slower for batched BF16 Qwen3.8-27B.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.