Reproduce “Dockerless: Environment-Free Program Verifier for Coding Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenhao Zeng, Yuling Shi, Xiaodong Gu, Chao Hu, Chaofan Wang, Yuhao Cui, Hongting Zhou, Mengnan Qi and 5 more
Dockerless is introduced as an environment-free program verifier that evaluates code patches by engaging in agentic repository exploration, eliminating the need for repository-specific execution environments. This system enables scalable post-training of coding agents, achieving resolve rates comparable to environment-based methods while demonstrating improved verifier performance on complex software engineering benchmarks.
Key insights
- Agentic repository exploration, involving question generation and evidence gathering via read-only shell tools, allows for deep, context-aware code verification without direct execution.
- Environment-free post-training with Dockerless can achieve performance parity with traditional environment-based methods, removing a major scalability bottleneck in training coding agents.
- Filtering agent trajectories based on Dockerless's correctness scores effectively selects high-quality examples for supervised fine-tuning, leading to significant performance gains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.