Reproduce “Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VI” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Abhinav Agrawal, Justin Domke
Researchers at the University of Massachusetts Amherst systematically analyzed factors influencing normalizing flow-based variational inference (flow VI) performance, providing a configuration "recipe" that achieves accuracy comparable to state-of-the-art Hamiltonian Monte Carlo (HMC) methods while requiring fewer sequential model evaluations, especially with parallel computing resources.
Key insights
- Real-NVP flows possess sufficient representational capacity for complex posteriors, with high capacity mitigating the need for complex, mode-spanning objective functions.
- Effective flow VI optimization relies heavily on reducing gradient variance, achieved through large batchsizes and the "sticking-the-landing" (STL) gradient estimator.
- Reliable step-size selection for flow VI optimization is critical, with a narrow range (10^-4 to 10^-3) consistently providing stable long-term performance, despite early-stage indicators potentially being misleading.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.