Reproduce “Patch correctness in automated program repair: A five-dimensional evidence framework and cross-benchmark audit from classical repair to coding agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuai Wu, Yanna Feng, Yufang Li, Xue Li, Zhijun Wang, Bolun Liu, Weilin Cai
The report presents a five-dimensional profile for evaluating patch correctness, tracking test provenance and exposure, behavioral scope, oracle basis, execution integrity, and label adjudication. It applies the profile to 33 contributions from 39 benchmark artifacts and proposes a reporting protocol. A prior PatchDiff audit found 51 of 77 sampled discrepancies uncertain; this is not a rate for all patches.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.