Reproduce “What Broke & Why: Practical Lessons from Reinforcement Learning Post-Training” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Luv Verma

The authors present a practical guide to reinforcement-learning post-training with verifiable rewards, organized around failures and diagnostic pivots. In a search-agent run, separately rewarded behaviors improved proxy scores while held-out exact-match accuracy fell from 26.8% to 19.9%. A smooth correctness gate improved optimization quality in ablations, but does not establish a universal fix and can worsen cold-start conditions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.