Reproduce “Causal Evaluation of Membership Inference Attacks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mathieu Even, Clément Berenfeld, Linus Bleistein, Tudor Cebere, Julie Josse, Aurélien Bellet
The authors introduce a causal framework for evaluating membership-inference attacks and a post-hoc estimator that reweights non-member scores using estimated membership propensities. In a controlled fine-tuning experiment on Pythia-70m, raw ROC AUC fell from 0.96 to 0.63 after correction, closer to the 0.58 same-distribution reference. The correction narrows, but does not eliminate, the gap.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.