Reproduce “Absolute Zero: Reinforced Self-play Reasoning with Zero Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu and 3 more
Absolute Zero Reasoner (AZR) uses a pretrained model to propose and solve its own checkable code tasks, learning from executable feedback without externally curated reasoning-task data. Its Python environment checks program validity, safety-sensitive imports, and repeatability, while restricting tasks to deterministic programs. On out-of-distribution math and coding benchmarks, AZR-Coder-7B scores 50.4 overall versus ORZ’s 48.6, under the reported evaluation protocol.
Key insights
- State-of-the-art reasoning capabilities can emerge in LLMs without any human-curated, domain-targeted data, challenging current data-hungry AI development.
- Autonomous curriculum generation within a verifiable environment fosters strong cross-domain generalization, as evidenced by code-trained models improving mathematical reasoning.
- Complex cognitive behaviors, such as ReAct-style planning and distinct task-specific reasoning strategies, naturally emerge from the self-play training process.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.