Reproduce “Variational Gaussian Process Diffusion Processes” on OpenResearch

About this paper

Prakhar Verma, Vincent Adam, Arno Solin

CVI-DP approximates posterior inference for nonlinear diffusion processes. It retains a linear-Gaussian posterior, representing observation and transition contributions as local exponential-family sites. For inference on the Double-Well prior at step size 0.01, five-fold cross-validated NLPD was 0.076 ± 0.186, compared with 0.069 ± 0.140 for SMC and 0.412 ± 0.231 for VDP. This result does not establish general superiority over SMC.

Key insights

• A site-based exponential family parameterization of the Gaussian variational posterior allows for faster, more stable inference by leveraging convex optimization and decoupling prior and observation contributions.
• By introducing a 'change of measure,' the inference problem with a non-linear DP prior can be transformed into an equivalent one with a tractable linear-Gaussian prior and new 'dense likelihood' terms.
• Adapting the base linear-Gaussian prior through posterior linearization within the M-step of VEM leads to a 'better objective' for learning the parameters of the non-linear prior diffusion.

Summary by alphaXiv.