Reproduce “CURE:Circuit-Aware Unlearning for LLM-based Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ziheng Chen, Jiali Cheng, Zezhong Fan, Hadi Amiri, Yunzhi Yao, Xiangguo Sun, Yang Zhang
CURE targets unlearning in language-model recommenders by identifying pathways involved in forgotten and retained examples and assigning different updates to forget-specific, retain-specific, and shared components. On GoodReads with GPT-2 Large, CURE-AT reports AUC 72.9, accuracy 70.6, and JSD 0.98, versus retraining’s AUC 74.2 and accuracy 70.85. This indicates near-retraining test utility and similar outputs on forgotten examples in this setting, not complete removal.
Key insights
- Unlearning and retention need not be optimized uniformly across the model: separating circuit components reduces interference between the two objectives.
- Graph structure can guide both causal circuit discovery and the construction of local retention buffers, linking recommender-system information with mechanistic interpretability.
- Activation patching offers more precise circuit localization, while first-order activation intervention provides a faster approximation suitable for reducing unlearning cost.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.