Reproduce “AutoEnvScaling: Automating the Data Flywheel with Terminal Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Simon Yu, Baolin Peng, Bo Liu, Zhengyan Shi, Zhaoyang Wang, Hao Cheng, Qianhui Wu, Wenlin Yao and 3 more
AutoEnvScaling makes terminal-environment design part of reinforcement learning: a proposer inspects solver failures, builds and tests tasks in a sandbox, then refreshes the training pool using solver feedback. Admission checks filter tasks but do not guarantee correct tests. On held-out Terminal-Bench 2.1, Qwen3.5-35B-A3B reaches avg@5 38.9 versus 34.6 with a matched static pool; finite-run results do not establish indefinite improvement.
Key insights
- Executable authoring and testing tools substantially improve the fraction of generated environments that pass admission checks compared with prompt-only and execution-feedback generation.
- Solver trajectories help shape the curriculum: removing them from the proposer’s workspace reduces downstream performance, while periodic calibration helps keep tasks useful as solver ability changes.
- A shared proposer–solver model can improve in finite training runs, but the results do not establish indefinite recursive improvement; environment checks also cannot guarantee that tests are free of shared flaws or shortcuts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.