Reproduce “Capacity Allocation at the Source: Sparse Target Optimization for LLM Knowledge Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dongyao Chen, Jiayun Lei, Zhiyuan Ji, Zhiying Deng, Wei Liu, Xiaobo Sun
STO improves locate-then-edit knowledge editing by adding an L1 penalty to MEMIT’s target-state objective, encouraging sparse changes to hidden representations before parameter updates. The authors link residual overlap to update interference and report lower residual inner products. On GPT-2 XL with 10,000 Multi-CounterFact edits, STO scores 86.9 versus MEMIT’s 80.4 in Success; this result does not establish preservation of unrelated behavior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.