Reproduce “TabPATE: Differentially Private Tabular In-Context Learning Without Public Data” on OpenResearch

About this paper

Dariush Wahdany, Matthew Jagielski, Jesse C. Cresswell, Adam Dziedzic, Franziska Boenisch

A PATE-style differentially private framework called TabPATE is introduced, enabling private in-context learning with Tabular Foundation Models without requiring public data. This approach effectively mitigates membership inference attacks by reducing their success to near-random levels, while maintaining competitive predictive utility, achieving 71.2% balanced accuracy at =10 on classification tasks.

Key insights

• Tabular in-context learning is empirically vulnerable to membership inference attacks, with active grey-box adversaries achieving up to 0.96 AUC.
• The structured nature of tabular data allows for the generation of effective synthetic queries for private aggregation, eliminating the dependency on public, in-distribution data.
• Allocating a small privacy budget ( ) to release differentially private marginal statistics for query generation improves predictive utility, especially for regression tasks.

Summary by alphaXiv.