Reproduce “Accepted but Wrong: Silent Failures of LLM Code Repairs after API Migrations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaobo Chen, Genliang Zhu, Ziyuan Wang, Chu Wang
The study evaluates whether release-note-derived test inputs improve release gates for runtime and API migrations. Across 52 affected test tasks, adding one author-written edge input reduced the task-mean share of wrong procedures released from 0.410 to 0.065, while false rejection stayed at 0.010. The measurements cover a finite repair-fragment menu; hard wrong repairs still passed at 0.168.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.