Reproduce “Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nat Dilokthanakul, Pedro A.M. Mediano, Marta Garnelo, Matthew C.H. Lee, Hugh Salimbeni, Kai Arulkumaran, Murray Shanahan
A Gaussian Mixture Variational Autoencoder (GMVAE) is presented for unsupervised clustering, extending standard VAEs with a Gaussian mixture prior over latent variables. The model successfully mitigates cluster degeneracy through a minimum information constraint, achieving a competitive unsupervised classification accuracy of 96.92% on MNIST and learning interpretable, disentangled latent representations for class and style.
Key insights
- The over-regularization problem in VAEs, particularly the regularization on the discrete latent variable, manifests as "cluster degeneracy," causing all data points to collapse into a single cluster.
- The minimum information constraint effectively delays or attenuates the regularization term's influence, allowing distinct clusters to form in the latent space before the term actively merges truly overlapping ones.
- The GMVAE learns disentangled latent representations where the discrete variable consistently controls the class of the generated data (e.g., digit type), and a continuous variable controls stylistic variations within that class.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.