Reproduce “From Hyperbolic Geometry Back to Word Embeddings” on OpenResearch

About this paper

Sultan Nurmukhamedov, Thomas Mach, Arsen Sheverdin, Zhenisbek Assylbekov

Researchers demonstrate that word-like representations can be derived from random points in hyperbolic geometry, which, when aligned to a vocabulary, perform competently in non-linear linguistic prediction tasks like Part-of-Speech tagging. This work, from institutions including Yandex School of Data Analysis and Nazarbayev University, suggests hyperbolic geometry provides an intrinsic structural foundation for word embeddings, outperforming random Euclidean baselines.

Key insights

• Hyperbolic geometry provides an intrinsic structural foundation from which word-like representations can emerge, suggesting semantic space inherently possesses negative curvature.
• The distribution of Pointwise Mutual Information (PMI) values from linguistic data shows structural similarity to the probability distribution of hyperbolic distances between random points, indicating a fundamental alignment.
• The effectiveness of random hyperbolic vectors in non-linear downstream tasks, contrasted with the poor performance of random Euclidean vectors, highlights the critical role of appropriate underlying geometry for word representations.

Summary by alphaXiv.