Reproduce “Sampling-Based Accuracy Testing of Posterior Estimators for General Inference” on OpenResearch

About this paper

Pablo Lemos, Adam Coogan, Yashar Hezaveh, Laurence Perreault-Levasseur

Researchers at Mila and Université de Montréal developed Tests of Accuracy with Random Points (TARP), a sample-based framework for validating posterior estimators. This method enables rigorous accuracy testing for generative models, effectively detecting biases that conventional high-posterior density methods fail to identify, and scales to high-dimensional parameter spaces.

Key insights

• The core theoretical insight is Theorem 3, proving that correct expected coverage probability, when assessed with positionable credible regions whose positions can depend on the observation, is a necessary and *sufficient* condition for verifying posterior estimator accuracy.
• Existing HPD-based coverage tests are insufficient because HPD regions are not positionable and can produce false positives, such as incorrectly validating uninformative estimators that simply return the prior distribution.
• TARP's ability to sample reference points `θr` with dependence on the observation `x` is crucial for its sufficiency property, enabling it to detect subtle biases and inaccuracies that prior methods cannot.

Summary by alphaXiv.