Reproduce “Learning Structured Representations with Hyperbolic Embeddings” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aditya Sinha, Siqi Zeng, Makoto Yamada, Han Zhao
HypStructure introduces a hyperbolic structured regularization approach for learning representations that explicitly embed label hierarchies, utilizing hyperbolic geometry to mitigate distortion. This method consistently achieves up to a 59.4% reduction in hierarchical distortion and improves Out-of-Distribution (OOD) detection performance by up to 2% AUROC across vision benchmarks, while also enhancing classification accuracy.
Key insights
- Hyperbolic geometry's negative curvature naturally accommodates the exponential growth of nodes in hierarchical structures, enabling low-distortion embeddings of trees compared to Euclidean space.
- Explicitly embedding the full label hierarchy, including internal tree nodes, in hyperbolic space significantly improves the fidelity of learned representations to the ground truth hierarchy.
- The learned hierarchical structure in feature space induces an interpretable eigenspectrum with 'phase transitions,' where eigenvalues correspond to different hierarchical levels, which is beneficial for OOD detection.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.