Reproduce “An Improved Variational Approximate Posterior for the Deep Wishart Process” on OpenResearch

About this paper

Sebastian Ober, Ben Anson, Edward Milsom, Laurence Aitchison

The authors introduce more flexible variational posteriors for deep Wishart processes by mixing rows, and optionally columns, of the Bartlett factor. In five-layer UCI Yacht regression, A-GW reaches test RMSE 0.37 ± 0.03 over 20 splits, versus 0.50 ± 0.04 for the earlier posterior and 0.57 ± 0.05 for a deep Gaussian process. This comparison changes the posterior family, and supports gains only in the tested setting.

Key insights

• The A-GW and AB-GW distributions are proper supersets of the original GW distribution, offering analytically and empirically demonstrated increased flexibility to model complex posterior shapes.
• By allowing arbitrary invertible matrices to linearly transform the Bartlett decomposition components, the A-GW and AB-GW distributions can capture dependency structures (e.g., non-central chi-squared components) that the GW distribution cannot.
• This increased flexibility in the approximate posterior enables DWPs to better leverage their intrinsic theoretical advantages, such as rotational invariance, translating into improved practical performance.

Summary by alphaXiv.