Reproduce “APo-VAE: Text Generation in Hyperbolic Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, Lawrence Carin, Jingjing Liu
APo-VAE, developed by researchers from Duke University and Microsoft Corporation, is a text generative model that utilizes hyperbolic latent spaces to capture natural language's hierarchical structure. The model achieves improved text generation quality by demonstrating lower perplexity on benchmarks and effectively mitigating posterior collapse in VAEs, while also enhancing performance in tasks like style transfer and dialogue generation.
Key insights
- Hyperbolic latent spaces can intrinsically model the hierarchical structure of natural language, allowing simpler sentences to cluster near the origin and more complex ones towards the periphery.
- Combining hyperbolic geometry with adversarial training and a data-driven prior (VampPrior) effectively mitigates posterior collapse in Variational Autoencoders for text.
- Implicit density matching via primal-dual KL divergence offers a stable and flexible way to train VAEs with complex, non-Euclidean distributions, avoiding explicit likelihood computations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.