Reproduce “Understanding Adversarial Training with Energy-based Models” on OpenResearch

About this paper

Mujtaba Hussain Mirza, Maria Rosaria Briglia, Filippo Bartolucci, Senad Beadini, Giuseppe Lisanti, Iacopo Masi

This research analyzes adversarial training through the lens of energy-based models, revealing the underlying energy landscape dynamics associated with robust and catastrophic overfitting. It introduces the Delta Energy Regularizer, a method shown to mitigate these overfitting phenomena, and demonstrates that robust classifiers can also function as generative models.

Key insights

• Classifiers can be interpreted as energy-based models, where logit outputs define an energy function, allowing for an energy-based analysis of adversarial training dynamics.
• Catastrophic and robust overfitting are linked to distinct patterns in the 'delta energy' (energy difference between natural and adversarial samples), with Abnormal Adversarial Examples serving as diagnostic indicators.
• Robust classifiers possess inherent generative capabilities, which can be leveraged to synthesize samples by guiding a sampling process with the classifier's energy function.

Summary by alphaXiv.