Reproduce “Long-Context Drafting Does Not Require Long-Context Drafters” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aref Jafari, Vansh Bhatia, Matt Qin, Subhajit Dutta Chowdhury, Ben Lee, Vikram Appia, Ali Ghodsi, Mehdi Rezagholizadeh
The method limits target-conditioned drafters to a recent attention window and re-bases rotary position indices, while the target retains full-context verification. Target-computed features from the full prefix still inform the drafter. In Inferact with Kimi-K3 on PG19, speedup was 2.51× at 8k and 2.25× at 2M; profitability remains conditional on speculative cost per emitted token.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.