Reproduce “Efficient and Robust Drafting with Multi-Token Prediction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Filippo Galgani, Ziwei Ji, Omri Homburger, Ganesh Jawahar, Michelle Ramanovich, Ivan Korotkov, Luciano Martins, Utku Evci and 4 more

G-MTP raises Gemma 4 31B throughput from 73.2 to 230.1 tokens per second, a 3.14× speedup. Its shared four-layer drafter uses target representations and reuses the target key-value cache; the target verifies drafts in parallel. The result was measured on 2×H100 GPUs at batch size 1 and does not establish memory or battery savings.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.