Reproduce “Automatic Perturbation Analysis for Scalable Certified Robustness and Beyond” on OpenResearch

About this paper

Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin and 1 more

Researchers from Northeastern University, Tsinghua University, UCLA, and Lawrence Livermore National Laboratory present an automatic framework for Linear Relaxation based Perturbation Analysis (LiRPA), making certified robustness applicable to arbitrary neural network architectures and diverse perturbation types. This framework, particularly with its "loss fusion" technique, drastically reduces the computational cost of certified training, allowing it to scale to large-scale datasets while providing provable guarantees on complex models.

Key insights

• Automating LiRPA makes certified robustness accessible to the broader machine learning community and applicable to state-of-the-art neural network architectures, akin to the impact of automatic differentiation for gradients.
• Bounding the scalar loss function directly through "loss fusion" yields a substantial asymptotic speedup for certified training, enabling scalability to datasets with hundreds or thousands of classes without sacrificing bound tightness.
• LiRPA can be leveraged beyond adversarial robustness to analyze fundamental properties of neural networks, such as the flatness of the optimization landscape, offering insights into generalization capabilities.

Summary by alphaXiv.