Reproduce “Privacy Auditing with One (1) Training Run” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Thomas Steinke, Milad Nasr, Matthew Jagielski
Researchers at Google propose and validate a novel scheme for privacy auditing that requires only a single training run, significantly reducing the computational cost of assessing privacy guarantees for differentially private machine learning models. The framework provides empirical lower bounds on privacy parameters by simultaneously varying multiple data points in one training process, yielding meaningful bounds for DP-SGD on CIFAR-10 with minimal accuracy reduction.
Key insights
- A rigorous theoretical framework, leveraging the connection between differential privacy and generalization, justifies privacy auditing by simultaneously varying multiple data points in a single training run.
- White-box auditing with carefully designed gradient attacks (Dirac canaries) can isolate the influence of individual canaries effectively, making their impact largely independent of the overall dataset size or number of iterations.
- Black-box auditing effectiveness is sensitive to the number of non-auditing examples and the overall dataset composition, indicating current black-box attack methodologies may be less robust in diverse data environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.