Reproduce “SIDE: Semantic ID Embedding for effective learning from sequences” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dinesh Ramasamy, Shakti Kumar, Chris Cadonic, Jiaxin Yang, Sohini Roychowdhury, Esam Abdel Rhman, Srihari Reddy
Meta Platforms developed a system that efficiently compresses and utilizes high-dimensional content embeddings from long user sequences in ad-ranking. The approach achieves a 2.4x improvement in Normalized Entropy gain and a 7.4x Return on Investment increase while reducing deployment costs by 20x.
Key insights
- Discrete-PCA (DPCA) demonstrates superior performance in maintaining reconstruction quality when fusing multiple content signals into a single, compact Semantic ID.
- Semantic ID Embedding (SIDE) effectively converts compact SIDs back into usable latent embeddings without requiring large, parameterized lookup tables, significantly reducing memory and inference costs.
- The combined application of VQ-fusion and SIDE in production-scale ad-ranking systems significantly improves predictive performance while achieving substantial cost efficiencies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.