Reproduce “Martingale Posterior Neural Processes” on OpenResearch

About this paper

Hyungi Lee, Eunggu Yun, Giung Nam, Edwin Fong, Juho Lee

Martingale Posterior Neural Processes (MPNPs) leverage martingale posterior distributions to model functional uncertainty in Neural Processes, inferring uncertainty directly from a neurally-defined predictive distribution. This approach consistently outperforms previous NP variants in 1D and high-dimensional regression, image completion, and Bayesian optimization tasks, achieving state-of-the-art log-likelihoods and improved normalized regret.

Key insights

• Martingale posteriors provide a theoretically sound and flexible framework for deriving functional uncertainty directly from a predictive distribution, bypassing the need for explicit prior specification.
• Generating pseudo contexts in a learned, latent feature space is crucial for the method's stability and performance, especially in handling high-dimensional and complex data where direct input generation is challenging.
• The approach naturally incorporates latent functional uncertainty paths, offering a more data-driven and flexible mechanism to capture epistemic uncertainty within the NP architecture.

Summary by alphaXiv.