Reproduce “Bayesian Hypernetworks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
David Krueger, Chin-Wei Huang, Riashat Islam, Ryan Turner, Alexandre Lacoste, Aaron Courville
Bayesian Hypernetworks introduce an efficient framework for approximate Bayesian inference in deep neural networks, employing invertible hypernetworks and weight normalization to capture expressive, multimodal posterior distributions over parameters. This approach yields competitive classification performance and provides superior uncertainty quantification for tasks like active learning, anomaly detection, and adversarial example detection.
Key insights
- Employing invertible generative models as hypernetworks provides a method to tractably compute the entropy term (`log q(θ)`) in the variational lower bound, a key challenge for general generative models.
- The re-parametrization of primary network weights using weight normalization, where the hypernetwork generates only scaling factors, effectively addresses scalability concerns, making complex Bayesian posteriors feasible for larger deep networks.
- Allowing for highly expressive approximate posteriors that capture multimodality and parameter correlations yields more robust and better-calibrated uncertainty estimates compared to simpler variational approximations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.