Reproduce “A Framework for Improving the Reliability of Black-box Variational Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Manushi Welandawe, Michael Riis Andersen, Aki Vehtari, Jonathan H. Huggins
Researchers from Boston University, Technical University of Denmark, and Aalto University developed the Robust and Automated Black-box Variational Inference (RABVI) framework to enhance BBVI's reliability and automation through principled learning rate adaptation and convergence detection. This framework consistently achieved user-specified accuracy targets across diverse models and datasets, reducing the need for manual tuning and making BBVI more accessible.
Key insights
- Robust convergence detection can be achieved by adaptively searching for an optimal window size that minimizes the maximum potential scale reduction factor for optimization iterates.
- The inherent bias characteristics of fixed-learning-rate stochastic gradient descent can be leveraged to estimate the error between the current variational approximation and the true optimal approximation.
- By dynamically adjusting the learning rate and the number of iterates used for averaging, the framework enables a principled balance between computational efficiency and the desired accuracy of the variational approximation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.