Reproduce “Large Language Models Can Self-Improve in Long-context Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siheng Li, Cheng Yang, Zesen Cheng, Lemao Liu, Mo Yu, Yujiu Yang, Wai Lam
SEALONG, a self-improvement method, enables large language models (LLMs) to enhance long-context reasoning by autonomously generating and learning from high-quality training data. This approach allows models like Llama-3.1-8B to achieve an absolute performance gain of 4.2 points, surpassing larger variants and even GPT-4o on aggregate long-context reasoning tasks without compromising short-context capabilities.
Key insights
- LLMs can generate effective self-supervision for long-context reasoning by leveraging internal consensus, where semantic consistency among multiple sampled outputs serves as a robust quality signal.
- MBR-based scoring, utilizing sentence embeddings to measure semantic consistency, is more effective at identifying high-quality reasoning trajectories than direct LLM self-evaluation or random selection.
- The self-improvement paradigm enables LLMs to 'unlock' their inherent reasoning potential, demonstrating significant performance gains even with relatively small amounts of self-generated training data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.