Reproduce “VERA: Scaling Verifiable Environments for Agentic co-Evolution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junqi Liu, Yongyang Pan, Zhuosong Jiang, Dongbai Li, Bo Zhang, Xitong Ling, Sheng Wang, Hanrong Ye and 9 more
VERA turns task trajectories into restartable sandboxes with evidence checks. It alternates updates to the reasoning model and procedural guide, holding one fixed during each update. In tested 9B comparisons, it scores 31.0 versus 20.7 on AutoCoWorkBench and 69.1 versus 56.1 on AutoMedBench, outperforming one-axis variants in both domains. These results support the combined approach in these settings, but do not establish verification alone caused gains.
Key insights
- Stage-level feedback makes workflow failures more actionable: agents can practice a specific step from a restored state rather than repeating an entire task.
- Model and harness improvements are complementary in the tested domains; neither the model-only nor harness-only variant matched the co-evolved system’s overall results.
- Performance and capability retention depend on scale: the 27B models generally retained or improved more general capabilities than the smaller 4B and 9B models, which showed declines on several benchmarks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.