Reproduce “A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gil Goldshlager, Nilin Abrahamsen, Lin Lin
Researchers at UC Berkeley and Lawrence Berkeley National Laboratory introduced SPRING, an optimizer designed to accelerate the training of neural network wavefunctions within the Variational Monte Carlo framework by combining subsampled natural gradient descent with a Kaczmarz-inspired projection technique. This method demonstrated faster convergence to lower energies and local energy variances across tested atomic and small molecular systems compared to existing optimizers like KFAC.
Key insights
- Incorporating historical optimization information via a Kaczmarz-inspired projection significantly improves convergence and accuracy compared to methods relying solely on the current minibatch.
- A decay factor (μ < 1) applied to the previous approximate solution is essential for stabilizing the optimization process and preventing numerical instabilities.
- The benefits of the Kaczmarz-inspired projection are distinct from those provided by norm constraints, which primarily enhance stability and allow for larger learning rates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.