Reproduce “AttenMIA: LLM Membership Inference Attack through Attention Signals” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pedram Zaree, Md Abdullah Al Mamun, Yue Dong, Ihsen Alouani, Nael Abu-Ghazaleh
AttenMIA, a new white-box framework, leverages internal attention mechanisms to perform membership inference attacks on large language models. This method significantly outperforms prior techniques, achieving up to 0.996 ROC AUC and 87.9% True Positive Rate at 1% False Positive Rate on various LLM architectures, while also enhancing training data extraction capabilities.
Key insights
- Self-attention mechanisms encode highly discriminative signals of training data membership, which can be systematically exploited for more effective privacy attacks.
- Memorized samples exhibit distinct attention patterns, including higher concentration (KL divergence to uniform) and more pronounced shifts under minor input perturbations compared to non-members.
- Common defenses like training data deduplication are largely ineffective against attention-based MIAs, indicating a need for new, attention-aware privacy-preserving strategies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.