Reproduce “Understanding and mitigating gradient pathologies in physics-informed neural networks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sifan Wang, Yujun Teng, Paris Perdikaris

This paper identifies gradient imbalance as a fundamental pathology in Physics-Informed Neural Networks (PINNs), where PDE residual gradients overshadow boundary condition gradients, leading to training instability. Researchers from the University of Pennsylvania address this issue by proposing an adaptive weighting scheme for loss components and an improved neural network architecture, achieving a 50-100x reduction in prediction error on benchmark problems.

Key insights

  • Gradient imbalance in Physics-Informed Neural Networks (PINNs), where PDE residual gradients overwhelm boundary condition gradients, is a fundamental cause of unstable and inaccurate predictions.
  • This gradient imbalance correlates with numerical stiffness in the optimization landscape, characterized by large eigenvalues of the Hessian matrix, leading to unstable training with standard optimizers.
  • Dynamically balancing the contributions of different loss terms and designing neural network architectures that reduce inherent stiffness can dramatically improve PINN accuracy and reliability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.