Reproduce “Differentially Private Permutation Tests” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ilmun Kim, Antonin Schrab
The paper presents a differentially private permutation test for comparing datasets. It privatizes the observed statistic and permutation threshold with a quantile-based mechanism whose noise scale is independent of permutations. Under exchangeability and global sensitivity, it controls type I error while preserving differential privacy. In a perturbed-uniform experiment, dpMMD outperforms alternatives in high- and mid-privacy settings, but is not universally superior.
Key insights
- Differentially private permutation tests can achieve finite-sample Type I error control and strong $(\epsilon, \delta)$-differential privacy simultaneously, with noise levels independent of the number of permutations.
- V-statistics (plug-in estimators) exhibit lower global sensitivity than U-statistics in private settings, making them more suitable for constructing privacy-preserving kernel tests with optimal power in high-privacy regimes.
- The intrinsic trade-off between privacy and statistical power can be precisely characterized, with dpMMD and dpHSIC achieving minimax optimal separation rates across different privacy regimes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.