Reproduce “Unbiased Bayes for Big Data: Paths of Partial Posteriors” on OpenResearch

About this paper

Heiko Strathmann, Dino Sejdinovic, Mark Girolami

Paths of Partial Posteriors estimates a chosen posterior expectation without first simulating from the full posterior. It computes estimates on increasingly large data subsets, then uses a randomly truncated, probability-weighted correction to recover the full-data expectation under stated conditions. In flight-delay prediction, debiasing achieved RMSE below 27.9 versus 32.6 for GP-SVI, but protocols and approximations differed.

Key insights

• The method achieves sub-linear average computational complexity (O(M * a * (N/a)^(1-α))) by using geometrically increasing batch sizes and a specific truncation probability distribution, making it scalable for large N.
• It maintains strict unbiasedness for posterior expectation estimates, avoiding the systematic errors common in many approximate large-data Bayesian inference techniques.
• The approach is general, not restricted by likelihood factorization, and can leverage existing MCMC methods on simpler partial posteriors, bypassing the need for complex, dataset-specific MCMC kernel design and tuning.

Summary by alphaXiv.