Reproduce “Quasi Monte Carlo methods enable extremely low-dimensional deep generative models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Miles Martinez, Alex H. Williams
QLVMs represent data with a low-dimensional grid of latent points and a decoder, replacing the learned encoder used by variational autoencoders. The method evaluates the decoder across the grid to estimate likelihoods and form embeddings. On three held-out, matched two-dimensional datasets, QLVM estimates exceed VAE and IWAE bounds, but higher-dimensional VAEs can perform better.
Key insights
- Direct marginal likelihood approximation via QMC integration is a viable and effective strategy for training deep generative models in extremely low-dimensional (2-3D) latent spaces, challenging the conventional view of its intractability.
- The underperformance of VAEs and IWAEs in very low dimensions is partly due to loose variational lower bounds, as their encoder networks often struggle to find accurate approximations to the true latent posterior, leading to poor training signals.
- QLVM's extremely low-dimensional embeddings are inherently interpretable, enabling direct visualization and analysis (e.g., clustering, smoothness via Jacobian norm) without the need for additional dimensionality reduction steps.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.