Reproduce “D$^2$Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ru Zhang, Renda Li, Ziyu Ma, Weijie Qiu, Chongyang Tao, Yong Wang, Xiangxiang Chu
D2Evo introduces a dual difficulty-aware self-evolution framework to enhance Large Language Model reasoning by dynamically selecting and generating appropriately difficult training samples. This method achieves up to 8.08% performance improvement over base models on mathematical reasoning tasks with significantly fewer real training samples and generalizes well to general reasoning benchmarks.
Key insights
- Maintaining a continuous supply of appropriately difficult training samples is crucial for efficient and stable Reinforcement Learning with LLMs, especially to counter effective data scarcity and dynamic difficulty shifts.
- Co-evolution between a question generation component and a problem-solving component can mutually enhance reasoning capabilities and deepen the understanding of problem structures.
- Anchoring generated training questions to real-world data and dynamically calibrating their difficulty to the learner's evolving capabilities significantly improves data efficiency and training stability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.