Reproduce “Half-Inverse Gradients for Physical Deep Learning” on OpenResearch

About this paper

Patrick Schnell, Philipp Holl, Nils Thuerey

Researchers at the Technical University of Munich introduce Half-Inverse Gradients (HIGs), a new optimization method for deep neural networks integrated with differentiable physics simulators. HIGs utilize a partial Jacobian inversion to balance optimization efforts across network parameters and physical outputs, enabling faster convergence and lower final loss values compared to standard optimizers like Adam on tasks such as nonlinear oscillator control, the Poisson problem, and quantum dipole control.

Key insights

• Effective optimization for physics-informed deep learning requires balancing the update mechanisms between the neural network's parameter space and the physical system's output space.
• A partial Jacobian inversion (specifically J^(-1/2)) provides a superior trade-off, mitigating vanishing/exploding gradients (common with first-order methods) and neuron oversaturation (seen with full inversions like Gauss-Newton).
• The proposed approach is generalizable across various physical systems and network architectures, demonstrating a principled way to handle the scale disparities in coupled neural network-physics training.

Summary by alphaXiv.