Reproduce “On Ranking Consistency of Pre-ranking Stage” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siyu Gu, Xiangrong Sheng
Researchers from Alibaba Group defined and addressed the problem of ranking inconsistency between pre-ranking and final ranking stages within large-scale e-commerce systems. They introduced the Ranking Consistency Score (RCS) and developed methods that improved online revenue per mille (RPM) by over 5% and click-through rate (CTR) by over 6%.
Key insights
- Ranking consistency directly correlates with online performance metrics like CTR and RPM in large-scale industrial systems.
- Both the individual ranking quality of single-objective models and their proxy-calibration (alignment of absolute prediction scores) are crucial factors influencing overall ranking consistency.
- Sample Selection Bias (SSB) significantly impacts the proxy-calibration of pre-ranking models, particularly on unexposed samples.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.