Reproduce “No Prior, No Leakage: Revisiting Reconstruction Attacks in Trained Neural Networks” on OpenResearch

About this paper

Yehonatan Refael, Guy Smorodinsky, Ofir Lindenbaum, Itay Safran

This work rigorously demonstrates that implicit-bias-driven reconstruction attacks on neural networks are fundamentally unreliable without strong prior knowledge about the training data, as the KKT-loss objective permits infinitely many global minima arbitrarily distant from the true data. It also reveals that networks trained more extensively, exhibiting stronger implicit bias, can be less susceptible to these specific attacks.

Key insights

• The implicit bias of trained neural networks, when leveraged for reconstruction without strong prior knowledge, does not uniquely identify the original training set, allowing for infinitely many alternative 'KKT-satisfying' data sets.
• Without specific prior information about the data domain, an attacker cannot reliably distinguish the true training set from an infinite number of vastly different alternatives that also satisfy the KKT conditions.
• Counter-intuitively, networks trained more extensively to achieve a stronger implicit bias (i.e., smaller deviation from KKT conditions) may be less susceptible to these reconstruction attacks, as the splitting budget for alternative solutions increases.

Summary by alphaXiv.