Reproduce “Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation” on OpenResearch

About this paper

Dmitry Bylinkin, Mikhail Aleksandrov, Savelii Chezhegov, Aleksandr Beznosikov

Researchers at Innopolis University and Moscow State University developed a new training framework for Physics-Informed Neural Networks (PINNs) by reformulating the optimization as a nonconvex-strongly concave saddle-point problem using Bregman divergence. This approach, called AdaptiveBGDA, significantly enhances training stability and achieved superior L2 Relative Error in 72.7% of 22 benchmark PDEs, outperforming existing state-of-the-art optimizers.

Key insights

• Reformulating PINN training as an N-SC SPP with Bregman divergence effectively mitigates conflicting gradients by adaptively balancing the contributions of different loss components.
• The use of Bregman divergence is crucial for properly handling the non-Euclidean geometry of the weight space, particularly when weights are constrained to a probabilistic simplex.
• The AdaptiveBGDA algorithm, combining adaptive optimization techniques within the Bregman proximal framework, offers a robust and generalizable solution for PINN training across diverse PDEs and architectures.

Summary by alphaXiv.