Reproduce “ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chengyi Yang, Renxu Bao, Jue Hou, Zongpei Teng, Ziheng Wang, Yujie Lin, Chengsong Huang, Junnan Zhu and 5 more
The authors present ProgramGym, which builds behavior-checked training environments for source-free program reproduction and pairs them with a staged agent workflow. The workflow separates exploration, implementation, and review, while persistent records carry evidence between stages. On ProgramBench, Qwen3.8-27B with Program-Agent scores 55.71% Tests Passed versus 34.87% with Claude Code, but resolves only 0.50% of tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.