Reproduce “Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jorge F. Urbán, Petros Stefanou, José A. Pons
The study by Urbán, Stefanou, and Pons demonstrates that enhancing the optimization process of Physics-Informed Neural Networks (PINNs) with self-scaled quasi-Newton methods and modified loss functions leads to substantially higher accuracy. Their work shows these improvements can reduce solution errors by orders of magnitude, making PINNs more competitive with traditional numerical methods even with compact network architectures.
Key insights
- Optimization algorithm choices, particularly the use of self-scaled quasi-Newton methods, fundamentally dictate the achievable accuracy and convergence speed of PINNs.
- Modified loss functions, like the logarithm of the MSE, improve PINN performance by effectively scaling gradients and influencing the Hessian matrix.
- Performance gains are attributed to improved conditioning of the Hessian matrix in the preconditioned parameter space, leading to more stable and faster convergence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.