Reproduce “Don't Forget Your Embeddings: Robust Knowledge Erasure via Precise Editing of Embeddings” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Clara Haya Suslik, Or Shafran, Mor Geva
EMBER extends persistent concept-erasure methods by subtracting selected concept-related directions from token embeddings while retaining other embedding components. In tests on Llama-3.1-8B-Instruct, adding EMBER to SNMF reduced multiple-choice concept accuracy from 43.8% to 29.1% while similar-domain and general scores remained close. After relearning, accuracy rebounded by 6 rather than 10 percentage points, indicating resistance to this probe, not irrecoverability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.