Reproduce “On the Consistency of Average Embeddings for Item Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Walid Bendada, Guillaume Salha-Galvan, Romain Hennequin, Thomas Bouabça, Tristan Cazenave
Researchers at Deezer Research and Université Paris Dauphine introduced a quantitative metric to assess how well average item embeddings represent their constituent items. Their analysis revealed a significant gap, with real-world music track embedding datasets demonstrating considerably lower consistency (e.g., 2-14%) compared to theoretical predictions (approaching 100%).
Key insights
- Theoretically, inner product similarities between an item and its collection's average embedding can be approximated by normal distributions as embedding dimension $d$ increases, leading to higher discriminative power.
- Ideal theoretical Consistency$_k$(X) remains high (approaching 100%) for small numbers of averaged items ($k$) but decreases with increasing $k$ and higher kurtosis of the embedding distribution.
- Real-world average embeddings exhibit substantially lower consistency (e.g., 2-14% on music datasets) than theoretically predicted, indicating that practical embeddings often do not possess the statistical properties assumed for high consistency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.