Reproduce “Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guanghao Li, Zihan Su, Hao Yu, Jinyang Jiang, Tao Ren, Zehao Li, Feng Lu, Ming Tang and 1 more
DAS-Wave decouples draft-computation depth from prefix-information depth in looped Transformers. It lets a shallow draft read mature representations of verified tokens, while carried parallel refinement builds proposals and a full-depth target verifies them. Across four checkpoints and four tasks, throughput speedup ranges from 4.00× to 6.96×; these measurements apply to greedy decoding in the tested single-GPU setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.