Reproduce “Semantic Retrieval at Walmart” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alessandro Magnani, Feng Liu, Suthee Chaidaroon, Sachin Yadav, Praveen Reddy Suram, Ajit Puthenputhussery, Sijie Chen, Min Xie and 3 more
A hybrid semantic retrieval system at Walmart.com combines traditional inverted indexes with neural network models to improve search relevance, particularly for complex "tail queries," demonstrating up to 20% recall improvement for these queries and a 5-10% increase in add-to-cart rates.
Key insights
- Combining traditional and neural retrieval leverages the strengths of both, handling rare tokens while improving semantic understanding and addressing vocabulary mismatch.
- Hard negative mining, especially through strategies like token matching, significantly enhances model performance by forcing the model to distinguish between highly similar but irrelevant products.
- Practical optimizations such as embedding dimension reduction (from 768 to 100-200 dimensions) and efficient index updates are crucial for deploying neural retrieval at scale without compromising quality or latency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.