Reproduce “Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Huzi Cheng, Zhewei Zhang
The authors compare five computation interfaces for graph reachability, where a model identifies which candidate lies downstream of a starting node. Four-layer GPTNeoX models train on paths of 3–6 hops and test on 7–12. Latent hidden-state interfaces lead on longer paths, while Chain-of-Thought performs worst. Transplants and circuit pruning support search in Bottleneck-latent, limited to small models on synthetic task.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.