Reproduce “Phantom Gains: Auditing Self-Improvement Against a Measured Null” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Cheng Xu, Nan Yan, Liming Chen, M-Tahar Kechadi
This research rigorously audits existing methodologies for evaluating language model self-improvement, identifying seven common measurement artifacts and statistical pitfalls that can lead to erroneous conclusions. It proposes robust statistical methods, including a threshold-free exact test, to reliably quantify capability expansion and corruption, revealing that self-training predominantly "sharpens" existing capabilities rather than "expands" into novel ones, while also causing substantial capability loss.
Key insights
- Single greedy decodes or underpowered evaluations can generate significant spurious "learnings" and "corruptions," even in a frozen, untrained model.
- Standard expansion statistics like ERm=1 exhibit a substantial non-zero null, incorrectly attributing gains where none exist, necessitating a more robust, threshold-free exact test.
- Self-training methods primarily "sharpen" an LLM's existing capabilities (improving consistency on problems it already has some capacity for) rather than "expanding" its ability to solve truly novel problems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.