Reproduce “Hard Examples Are All You Need: Maximizing GRPO Post-Training Under Annotation Budgets” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Benjamin Pikus, Pratyush Ranjan Tiwari, Burton Ye
Research demonstrates that fine-tuning large language models with Group Relative Policy Optimization (GRPO) is most effective when prioritizing "hard" examples, where the base model initially struggles. This strategy consistently yields substantial performance gains, such as a 39.42% absolute increase for Qwen3-14B on GSM8K, and also improves generalization to out-of-distribution problems.
Key insights
- Prioritizing 'hard' examples, where the base model frequently fails, consistently leads to the greatest performance improvements in GRPO fine-tuning.
- Hard examples maintain a higher 'learnable percentage' by sustaining within-group reward variance, which is essential for GRPO's contrastive learning mechanism.
- Training on challenging problems also fosters improved generalization to significantly harder, out-of-distribution problem sets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.