Reproduce “Hamiltonian Variational Auto-Encoder” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anthony L. Caterini, Arnaud Doucet, Dino Sejdinovic
HVAE uses joint-likelihood gradients to guide latent samples through a smooth, time-inhomogeneous trajectory, forming a differentiable posterior without a learned reverse process. In Gaussian parameter-estimation tests, it performs best as dimension increases, and tempering generally lowers error. On binarized MNIST, a representative setting reports estimated test NLL of 82.62 nats versus 83.20 for the base VAE; gradient storage may require substantial memory.
Key insights
- Deterministic, time-inhomogeneous Hamiltonian dynamics can be integrated into VAEs to produce unbiased ELBO estimators that are fully compatible with the reparameterization trick.
- Optimal reverse kernels in MCMC-based ELBO estimation are implicitly managed through the use of deterministic transformations, simplifying the theoretical framework and enhancing robustness.
- Combining target-informed Hamiltonian dynamics with tempering offers a more expressive approximate posterior and improved inference capabilities compared to purely target-agnostic flow-based models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.