Reproduce “Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh, Nanyun Peng, Kai-Wei Chang, Yong Jae Lee
TEV-GYM builds long-horizon terminal tasks from hidden, known-correct worlds and uses those worlds to verify final outputs and score intermediate progress. Its progressive reward recipe trains agents on 1,170 environments across seven domains. On 18 Linux-container-compatible ALE-CM tasks, TEV-27B reaches a best-of-five pass rate of 0.28, matching frontier models on this metric, while remaining behind them on harder transfer tasks.
Key insights
- A shared latent representation can link task construction, input generation, completion checking, and progress measurement, reducing inconsistencies between these components.
- Dense progress feedback can make group-relative RL trainable on tasks where terminal-only rewards yield mostly uninformative all-failure groups.
- PPRS intentionally rewards incomplete but correct progress, so it changes the training objective rather than serving as policy-invariant potential-based shaping.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.