Reproduce “rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yifei Liu, Li Lyna Zhang, Yi Zhu, Bingcheng Dong, Xudong Zhou, Ning Shang, Fan Yang, Mao Yang
Microsoft Research Asia introduces a new dataset and methodology, `rStar-Coder`, which significantly enhances the code reasoning capabilities of large language models (LLMs) for complex competitive programming tasks. This approach enables smaller models, such as `rStar-Coder-7B`, to achieve state-of-the-art performance, outperforming models like GPT-4o on LiveCodeBench, by leveraging a meticulously verified, large-scale dataset of 418K problems.
Key insights
- The quality and rigorous verification of training data are more crucial than model size or sheer dataset quantity for improving competitive code reasoning performance.
- A mutual verification process can reliably generate ground-truth labels for synthetic code problems, achieving 96.8% accuracy in output labeling.
- Effective test case generation requires a structured approach to produce diverse, constraint-satisfying, and complexity-varying inputs, which direct LLM prompting struggles to achieve.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.