Reproduce “Optimising Distributions with Natural Gradient Surrogates” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jonathan So, Richard E. Turner
SNGD optimizes a target probability objective using the natural-gradient geometry of a tractable surrogate distribution. For exponential-family surrogates, it obtains updates from autodiff gradients in dual mean and natural parameterizations, without explicitly inverting a Fisher matrix. On an 82-observation sheep tick-count task, it outperformed gradient descent, BFGS, and target NGD even under matched parameterization; this is a single example.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.