Reproduce “LiST: Lipschitz Scaling Training for Robust and Calibrated Neural Networks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Arthur Chiron, Franck Mamalet, Thomas Massena, Thomas Deltort, Mathieu Serrurier

This work introduces Lipschitz Scaling Training (LiST), a dynamic training strategy that unifies certified robustness and intrinsic calibration in neural networks. LiST automatically identifies an optimal Lipschitz constant (L*) by leveraging a newly established duality between the Lipschitz constant and temperature scaling, yielding models that are calibrated out-of-the-box, achieve competitive accuracy and robustness, and improve sample efficiency by safely re-integrating calibration data.

Key insights

  • There is a fundamental functional equivalence, or duality, between a neural network's global Lipschitz constant and the temperature scaling parameter used for calibration.
  • Calibration can serve as an unsupervised criterion to dynamically discover an intrinsic Lipschitz constant (L*) that inherently yields a well-calibrated network without manual tuning or post-hoc adjustment.
  • By incorporating a loss offset parameter (xi), models can navigate the accuracy-robustness Pareto front while continuously maintaining intrinsic calibration, decoupling these concerns.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.