Reproduce “Posterior Collapse and Latent Variable Non-identifiability” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yixin Wang, David M. Blei, John P. Cunningham

LIDVAE proves exact posterior collapse occurs if and only if the likelihood is constant across latent values for a specified dataset and parameter setting. Its decoder uses bijective Brenier maps and a full-rank coordinate injection to make the likelihood injective. On nine-layer Fashion-MNIST, it reports AU 1.0, approximate MI 2.6, and test log likelihood −242.3, versus 0.6, 2.4, and −245.3 for β-VAE, without establishing universal superiority.

Key insights

  • Posterior collapse is not merely an artifact of deep learning or approximate inference but an intrinsic property linked to latent variable non-identifiability within the generative model.
  • This equivalence holds true even in classical probabilistic models with exact inference, highlighting the generality of the non-identifiability problem.
  • Designing generative models with provably identifiable latent variables offers a principled, model-centric approach to prevent posterior collapse, rather than relying on heuristic adjustments to the inference objective.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.