Reproduce “FastDraft: How to Train Your Draft” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ofir Zafrir, Igor Margulis, Dorin Shteyman, Shira Guskin, Guy Boudoukh
FastDraft trains compact draft models for speculative decoding when a model family lacks a suitably small compatible model. It pre-trains on general text, continues training with code where needed, then fine-tunes on target-generated sequences. A 50M-parameter Phi-3-mini draft reached acceptance rate 0.369 on CNN-DailyMail after fine-tuning; device TPOT speedups were 1.5× and 2× on two tasks, within tested English settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.