Reproduce “Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Emile Mathieu, Charline Le Lan, Chris J. Maddison, Ryota Tomioka, Yee Whye Teh
The authors develop a variational autoencoder with a Poincaré-ball latent space, hyperbolic normal distributions, and a geometry-aware gyroplane decoder for hierarchical data. On MNIST, the gyroplane decoder improves test marginal log-likelihood over an ordinary multilayer perceptron, with gains diminishing at higher latent dimensions. On the CS PhDs link-prediction test, its five-dimensional model scores 59.8 ± 1.2 AUC versus 56.5 ± 1.1 for the Euclidean counterpart.
Key insights
- Hyperbolic latent spaces, specifically the Poincaré ball model, naturally accommodate data with hierarchical structures due to their exponential volume growth, offering more faithful and efficient embeddings than Euclidean spaces.
- Rigorous mathematical treatment of hyperbolic probability distributions (Riemannian normal and wrapped normal) and their reparametrizable sampling schemes is essential for enabling gradient-based optimization of hyperbolic VAEs.
- Incorporating a geometry-aware decoder, like the proposed gyroplane layer, is crucial for the model to effectively leverage the properties of the hyperbolic latent space.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.