Reproduce “CopyLoop: Fast Looped Transformer Decoding via Context Copying and Shallow Screening” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Wenyu Du, Stephen Chung

CopyLoop accelerates decoding in recurrent language models by copying multi-token continuations from earlier context, screening them at shallow depth, and overlapping surviving tokens with deeper verification. On Base Raven-TinyLlama HumanEval+, it reports 10.70× decode-throughput speedup over autoregressive decoding, compared with 6.55× for LoopSpec and 5.44× for WaveFront, with 12.20% accuracy for all; these BF16 results cover one task and model.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.