Reproduce “SharpDraft: Accelerating Long-Context Speculative Decoding with Cardinality-Aware Query Scaling” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jeonghoon Park, Seongwoon Jo, Jongwon Lee, Taesik Gong
The authors introduce SharpDraft, which scales a draft model’s Query vectors as the number of visible Keys grows, without changing model weights or rebuilding cached Keys and Values. In Qwen3-8B with DFlash, it reaches 2.675× overall end-to-end speedup, versus 2.144× for the best tested LoRA setting; this result covers three benchmarks under greedy decoding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.