Reproduce “InfuseLoop: Cross-Trajectory Latent Injection for Scalable Parallel Looped Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhaofeng Sun, Lanxiang Hu, Yu-Yang Qian, Jiaqi Leng, Yuanbo Yang, Chi Zhang, Youyou Lu, Peng Zhao and 1 more
InfuseLoop improves parallel looped Transformers by routing causally available states from other prediction paths into deeper loop updates while preserving the overlapped schedule. In roughly 0.6B-parameter models trained on 6B tokens, InfuseLoop-8 scores 43.07% on a seven-task average, versus 42.23% for PLT-8. It adds 0.3% decoding time over PLT; sequential LT-8 scores higher.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.