Reproduce “SpecSearch: Towards Hardware-Aware Efficient Speculative Decoding” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Selin Yildirim, Ali Zeyrek, Deming Chen

SpecSearch selects decoding policies for each workload and GPU: autoregressive or speculative decoding, draft length, and precision. A cost model filters infeasible settings, and full calibration measures candidates on the deployment GPU; a learned selector predicts from calibration data on other GPUs. Across four GPUs, full calibration averages 2.01× AR and 1.36× vLLM SD throughput, using Qwen3-4B with DFlash.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.