Reproduce “Pre-trained Language Model for Web-scale Retrieval in Baidu Search” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiding Liu, Guan Huang, Jiaxiang Liu, Weixue Lu, Suqi Cheng, Yukun Li, Daiting Shi, Shuaiqiang Wang and 2 more
Baidu Inc. developed and deployed an ERNIE-based Pre-trained Language Model (PLM) system for web-scale retrieval, enhancing semantic matching capabilities within Baidu Search. The system, utilizing a multi-stage training paradigm, improved Recall@10 for tail queries by 8.1% offline and led to a 0.31% increase in online click behaviors and a 0.62% increase in average post-click dwell time.
Key insights
- A progressive multi-stage training paradigm effectively adapts generic PLMs to domain-specific retrieval tasks, significantly boosting performance and generalization, particularly for tail queries.
- The combination of poly-attention during training and efficient embedding compression with quantization allows state-of-the-art PLMs to be effectively and efficiently deployed in web-scale search systems.
- Integrating semantic retrieval in parallel with conventional methods, along with a unified post-retrieval filtering module, provides a robust framework for enhancing large-scale search engines.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.