Reproduce “Riemannian Laplace approximations for Bayesian neural networks” on OpenResearch

About this paper

Federico Bergamin, Pablo Moreno-Muñoz, Søren Hauberg, Georgios Arvanitidis

Researchers at DTU Compute developed Riemannian Laplace Approximations (RIEM-LA) for Bayesian Neural Networks, a method that leverages a loss-aware Riemannian metric to define a more accurate approximate posterior. This approach consistently outperforms conventional Laplace approximations in uncertainty quantification and predictive performance, while also demonstrating robustness to prior choice.

Key insights

• A simple parametric Gaussian approximation can effectively represent a complex, locally non-Gaussian posterior if it is defined in a geometrically informed tangent space and then non-linearly mapped back via geodesics.
• By endowing the parameter space with a Riemannian metric that intrinsically encodes the local steepness and direction of the loss landscape, the approximate posterior inherently adapts to the true posterior's shape.
• The exponential map, by tracing shortest paths (geodesics) on the loss manifold, naturally guides samples into regions of low negative log-posterior, even if the initial tangent vector might point to a high-loss region in Euclidean space.

Summary by alphaXiv.