Reproduce “Finding the Pivots to Repair Broken Source-Test Evolution Using LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiao Chen, Songqiang Chen, Jialun Cao, Yun Peng, Chun Yong Chong, Shing-Chi Cheung
The authors evaluate how language models repair tests after intentional source changes across 500 dynamically validated cases from 15 Java projects. They compare failure logs, changed code, relevant unchanged code, and repair strategies. With all three context types, Codex made 20/27 correct repairs, up from 12/27, on a selected challenging subset; this result does not establish performance on all test repairs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.