Reproduce “Membership Inference Attacks against Language Models via Neighbourhood Comparison” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Sachan, Taylor Berg-Kirkpatrick

Researchers developed "neighbourhood attacks," a novel reference-free Membership Inference Attack (MIA) for language models that generates synthetic "neighbour" samples to calibrate loss scores. This approach substantially outperforms existing reference-free attacks and Likelihood Ratio Attacks (LiRAs) under realistic adversarial data assumptions, while demonstrating that Differential Privacy remains an effective countermeasure.

Key insights

  • The performance of state-of-the-art LiRAs is highly fragile and critically dependent on the quality and domain relevance of data used for training reference models; small deviations significantly degrade their effectiveness.
  • Effective difficulty calibration for MIAs can be achieved without external reference models by leveraging the target model's own responses to synthetically perturbed "neighbour" samples.
  • Subtle, single-word lexical perturbations are more effective for generating "neighbour" samples than multiple changes, as they preserve semantic context while creating non-training adjacent data points that highlight overfitting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.