Reproduce “Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Toan Tran, Ruixuan Liu, Li Xiong

DuoLearn, a dual-purpose training method for Large Language Models, mitigates Membership Inference Attacks by dynamically identifying and applying gradient ascent to "memorized tokens" while performing standard learning on "hard tokens." This approach achieves robust privacy protection, with MIA AUC often below 0.55, while significantly reducing utility degradation compared to Differential Privacy, demonstrating a 15-27% perplexity improvement over DPSGD on the Wikipedia dataset.

Key insights

  • Membership Inference Attacks in LLMs are deeply linked to token-specific characteristics; tokens predicted with unusually high confidence serve as strong indicators of data membership.
  • A dynamic, token-level dual-purpose training approach can effectively balance model utility and privacy by simultaneously promoting learning on challenging tokens and actively unlearning strong predictions on memorized ones.
  • An accurately calibrated, in-domain reference model is crucial for precisely identifying and targeting memorized tokens for unlearning, ensuring effective privacy protection without excessive degradation of model performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.