Reproduce “Abstain and Validate: A Dual-LLM Policy for Reducing Noise in Agentic Program Repair” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
José Cambronero, Michele Tufano, Sherry Shi, Renyao Wei, Grant Uy, Runxiang Cheng, Chin-Jung Liu, Shiying Pan and 2 more
The authors evaluate two filters for agentic program repair: bug abstention before repair and patch validation after candidates are generated. On 174 curated human-reported Google bugs, specification-based validation improves filtered fail-to-pass@1 over build-and-test checks and a no-spec judge. Combined strict filtering reaches 53%, but only 12 bugs retain a patch; this is a held-out-test result, not a production success rate.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.