Reproduce “When Automated Program Repair Meets Regression Testing -- An Extensive Study on 2 Million Patches” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiling Lou, Jun Yang, Samuel Benton, Dan Hao, Lin Tan, Zhenpeng Chen, Lu Zhang, Lingming Zhang
An extensive empirical study investigates the impact of Regression Test Selection (RTS) on Automated Program Repair (APR) efficiency, demonstrating that integrating RTS significantly reduces test executions during patch validation. The research analyzed over 2 million patches from 12 APR systems on Defects4J, showing that method-level and statement-level RTS offer substantial efficiency gains, with reductions up to 99.95% for plausible patches.
Key insights
- Measuring APR efficiency solely by patch count is misleading; the number of test executions offers a more accurate metric, and inconsistent RTS configurations skew comparisons between systems.
- Regression Test Selection (RTS) is a highly effective, yet largely overlooked, optimization for APR, significantly reducing test validation costs.
- Finer-grained RTS (method-level and statement-level) consistently outperforms class-level RTS, and while test selection is paramount, combining it with test prioritization can offer further, albeit smaller, efficiency gains.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.