Reproduce “Reasoning to Learn from Latent Thoughts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yangjun Ruan, Neil Band, Chris J. Maddison, Tatsunori Hashimoto
Researchers at Stanford University, the University of Toronto, and the Vector Institute introduced "Reasoning to Learn," a framework that enhances large language model data efficiency by explicitly modeling and leveraging latent thought processes during pretraining. Their Bootstrapping Latent Thought (BoLT) algorithm allows models to iteratively improve their ability to generate and learn from high-quality latent thoughts, achieving 25.38% on MATH and outperforming strong baselines by a large margin.
Key insights
- Explicitly modeling latent thoughts in a separate, dedicated space is crucial for data-efficient learning, outperforming methods that intersperse reasoning directly within the raw text.
- Language models can iteratively bootstrap their own ability to infer and utilize high-quality latent thoughts through an EM-like self-improvement process, moving beyond initial external supervision.
- Increased inference compute during the E-step (latent generation) directly correlates with higher-quality synthetic latents and improved downstream model performance, offering a new avenue for scaling data efficiency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.