Reproduce “Flat-RLVR: Sharpness-Aware Reinforcement Learning with Verifiable Rewards for Reasoning Capacity in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yeonbum Kim, Hyun Kyu Lee, Sung Whan Yoon
Flat-RLVR uses sharpness-aware updates during reinforcement learning with verifiable rewards to reduce concentration on already-likely answers while retaining less-common correct responses. On Qwen2.5-Math-1.5B across five in-domain math benchmarks, Flat-GRPO averages 67.3% Pass@256 versus GRPO’s 64.8%, while Pass@1 is 15.2% versus 15.6%. These are means over three seeds; the result measures sampled coverage through 256 attempts, not general reasoning ability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.