Reproduce “MagicBeanStalk: Escaping the RLVR Zero Advantage Trap with Recursive Data Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ximing Lu, Aviral Kumar, Jan Kautz, Yejin Choi, David Acuna
MagicBeanStalk uses verified solutions to build math problems preserving targeted reasoning, then screens them for mixed outcomes under a target policy and response budget. In a 78,000-problem corpus, about 88% were all-correct or all-incorrect across eight gpt-oss-120b rollouts at 32K. For gpt-oss-120b, training raised average pass@1 across eight benchmarks from 61.2% to 69.8% at 16K.
Key insights
- A problem’s training usefulness is not intrinsic: it depends on the target policy, number of sampled responses, and response-length limit.
- Hard problems can serve as specifications for new training tasks: their solutions reveal reasoning blueprints even when the target model currently cannot solve them.
- Filtering and synthesis play different roles: filtering selects useful examples already present, while synthesis can create new examples targeting underrepresented skills.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.