Reproduce “RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Honglei Zhuang, Zhen Qin, Rolf Jagerman, Kai Hui, Ji Ma, Jing Lu, Jianmo Ni, Xuanhui Wang and 1 more
Google Research developed RankT5, a framework that adapts T5 sequence-to-sequence models for text ranking by enabling them to directly output numerical scores and be fine-tuned with ranking-specific losses. This approach improved MRR@10 by over +1.8% on MS MARCO and +2.8% on Natural Questions compared to prior T5 adaptations, also demonstrating enhanced zero-shot generalization on out-of-domain datasets.
Key insights
- Listwise ranking losses, such as Softmax and Poly1, consistently yielded superior ranking performance compared to pointwise and pairwise losses, highlighting the benefit of optimizing for relative document order.
- Both the full encoder-decoder and encoder-only T5 architectures achieved comparable ranking performance, suggesting that the T5 decoder's generative capabilities may not be essential for text ranking when fine-tuning data is plentiful.
- Larger T5 models consistently improved ranking scores, and models trained with listwise losses exhibited better zero-shot generalization to unseen domains, particularly in specialized fields.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.