Reproduce “Adversarial Feature Alignment: Balancing Robustness and Accuracy in Deep Learning via Adversarial Training” on OpenResearch

About this paper

Leo Hyun Park, Jaeuk Kim, Myung Gyo Oh, Jaewoo Park, Taekyoung Kwon

Adversarial Feature Alignment (AFA) introduces a fully-supervised adversarial contrastive learning approach to resolve the robustness-accuracy tradeoff in deep learning models. This method notably improves performance, achieving 91.01% clean accuracy and 52.05% AutoAttack robust accuracy on CIFAR10 (ResNet-18), surpassing existing techniques by enhancing clean accuracy while preserving strong adversarial robustness.

Key insights

• The robustness-accuracy tradeoff is primarily caused by feature space misalignment, where existing adversarial training methods often fail to create well-clustered and separated class features.
• The concept that 'separation does not imply alignment' is introduced, arguing that mere class separation in the input space is insufficient if intra-class samples are still poorly organized in feature space.
• Using benign samples as a stable, correctly labeled 'contrastive view' in fully-supervised adversarial contrastive learning effectively guides feature alignment and prevents 'class collision problems' observed in self-supervised methods.

Summary by alphaXiv.