Reproduce “A Statistical and Multi-Perspective Revisiting of the Membership Inference Attack in Large Language Models” on OpenResearch

About this paper

Bowen Chen, Namgi Han, Yusuke Miyao

Researchers from The University of Tokyo statistically revisit Membership Inference Attacks (MIAs) in Large Language Models (LLMs), conducting thousands of experiments to identify factors like model size, data domain, and text features that drive MIA performance and to explain observed inconsistencies. The study provides a robust, multi-perspective analysis of MIA effectiveness, revealing the conditions under which attacks succeed or fail and the underlying model dynamics.

Key insights

• MIA performance is highly dependent on factors such as LLM size (improving with larger models), data domain (e.g., Wikipedia and FreeLaw showing higher performance), and the specific method used to construct member/non-member datasets.
• Despite generally low average MIA performance, a 'notable amount' of differentiable member and non-member outlier instances exist, accounting for previous successful MIA reports and highlighting that success is often localized.
• Effective threshold decision-making is an overlooked challenge; thresholds are highly unstable and vary significantly across domains and model sizes, hindering real-world application and generalization of MIAs.

Summary by alphaXiv.