Reproduce “Localizing Paragraph Memorization in Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Niklas Stoehr, Mitchell Gordon, Chiyuan Zhang, Owen Lewis
The authors study how GPT-Neo 125M recites memorized PILE paragraphs. They perturb prefix tokens, analyze gradients and attention, and update selected weights. After 10 gradient steps, tuning the top-gradient 0.1% of weights lowers mean exact match on memorized paragraphs to 26.75, near 26.90 with all weights and below 49.80 with random weights. This one-model result does not establish erasure.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.