Reproduce “Unifying and Beyond the Theory of Pervasive Capability Degradation in LLM Post-training SFT from Feature Interference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hang Chen, Jiaying Zhu, Wenya Wang
AEGIS monitors activation-margin drift toward competing token boundaries while retaining cross-entropy on target and pervasive anchor data. The authors’ theory links feature interference under narrow supervised fine-tuning to changed token predictions, subject to assumptions. On Qwen2.5-7B GSM8K, it scores 0.739 target accuracy and 0.683 P-Acc., versus 0.714 and 0.668 for Vanilla; this advantage is not universal across metrics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.