Reproduce “Private-Shared Disentangled Multimodal VAE for Learning of Hybrid Latent Representations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mihee Lee, Vladimir Pavlovic
The Disentangled Multi-modal Variational Autoencoder (DMVAE) learns separate private and shared latent representations from multimodal data, effectively accommodating both continuous and discrete latent factors. This approach enabled the model to achieve 77.4% accuracy on MNIST semi-supervised classification with minimal paired data, outperforming baselines and demonstrating improved interpretability through disentangled representations.
Key insights
- Explicitly disentangling shared and private latent spaces improves interpretability and generative capabilities in multimodal learning.
- The Product-of-Experts approach effectively fuses information from multiple modalities into a consistent shared representation, robust to missing data.
- Integrating Gumbel SoftMax enables a single framework to handle both continuous and discrete underlying factors, crucial for real-world hybrid data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.