Reproduce “Achieving High Accuracy with PINNs via Energy Natural Gradients” on OpenResearch

About this paper

Johannes Müller, Marius Zeinhofer

This work introduces Energy Natural Gradient Descent (E-NGD), an optimization algorithm that enables Physics-Informed Neural Networks (PINNs) and the Deep Ritz Method (DRM) to achieve relative L2 errors ranging from 10^-6 to 10^-8, outperforming conventional optimizers that typically stall at 10^-3 to 10^-4. E-NGD significantly reduces the total computational time needed to reach high accuracy across diverse PDE problems.

Key insights

• The energy natural gradient in parameter space is directly linked to the Newton direction in function space, which powerfully guides the neural network towards the true solution of the PDE.
• The specific choice of an energy-based metric for the natural gradient is crucial; other natural gradient formulations like Hilbert Natural Gradient Descent (H-NGD) often fail to converge or achieve high accuracy.
• Despite higher per-iteration computational costs due to O(p^3) complexity, E-NGD's dramatic reduction in the number of required iterations leads to overall faster training times for achieving high-accuracy solutions.

Summary by alphaXiv.