Reproduce “Library of Layers (LoL): Adaptively Ordering Transformer Layers at Runtime” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Abdellah Rahmani, Sean McLeish, Ang Li, Sanae Lotfi, Pascal Frossard, Micah Goldblum
LoL lets a router select and reuse complete transformer blocks during inference, rather than executing every block in a fixed order. A top-1 router chooses blocks for each window from a shared library, and selected blocks may repeat. On synthetic mazes, forcing a learned route yields 99.975% on a strategy-prefix measure, not final-answer accuracy or evidence of general reasoning control.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.