Reproduce “Variational Inference via Entropic Transport Descent” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vincent Pacelli, Akash Ratheesh, Evangelos Theodorou
A new family of particle-based variational inference algorithms, Entropic Transport Descent (ETD), is presented, which leverages entropic optimal transport to improve the approximation of intractable target distributions. ETD achieves near-perfect variance recovery and superior multimodal coverage compared to existing methods, notably reducing total variation distance on challenging high-dimensional molecular Boltzmann distributions like Lennard-Jones 13 from ~0.4 to ~0.053.
Key insights
- Reformulating the JKO proximal scheme by lifting the optimization to the space of couplings and applying the KL chain rule makes it amenable to efficient computation via entropy-regularized optimal transport.
- The stationary distribution of balanced, score-free ETD is analytically characterized as biased, but this bias can be precisely eliminated using importance-corrected target weights, avoiding Metropolis-Hastings steps.
- A family of τ-ETD coupling regimes provides flexibility, allowing adaptation to different inference challenges by controlling the enforcement of the target marginal during particle updates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.