Reproduce “Scaling and Stabilizing Large-Scale Embedding-Based Retrieval” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhen Yang, Juexin Lin, Hongwei Shang, Kaihao Li, Feng Liu, Satya Chembolu, Xunfan Cai, Xinyi Liu and 3 more

Walmart Global Tech introduced a framework for large-scale embedding-based retrieval systems, combining hybrid negative mining with legacy-aware warm-start distillation. This approach led to a 7.34% improvement in NDCG@5 and a 0.50% increase in gross revenue during live A/B tests, by enhancing training signal quality and ensuring stable model transitions to higher-capacity encoders.

Key insights

  • Scaling the diversity and quality of negative examples through hybrid online and offline mining significantly enhances a dual-encoder model's discriminative power and bridges the training-inference gap.
  • Preserving accumulated domain knowledge via warm-start distillation is essential for stable and successful transitions to higher-capacity encoder architectures, preventing performance regressions typically associated with 'cold' model upgrades.
  • A joint training objective balancing engagement and relevance, coupled with robust negative sampling, enables models to effectively learn both user behavior and semantic matching.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.