Reproduce “Feedback Shortcut: Speculative Decoding for Latent Feedback Transformers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Benhao Huang, Xuezhe Ma, Hector Liu

Feedback Shortcut enables speculative decoding for latent-feedback Transformers. It aligns multi-token prediction head states as feedback stand-ins, then recomputes pending positions during the next verification call. For a 780M model with three heads, one correction, and batch size 1, it reports 0.8287 BPB versus 0.8288 for exact decoding, and 2.75× throughput; this is single-GPU steady-state serving, not exact output equivalence.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.