Reproduce “ListT5: Listwise Reranking with Fusion-in-Decoder Improves Zero-shot Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Soyoung Yoon, Eunbi Choi, Jiyeon Kim, Hyeongu Yun, Yireun Kim, Seung-won Hwang
LISTT5, a listwise reranking framework, improves zero-shot retrieval performance by leveraging a Fusion-in-Decoder architecture and an m-ary tournament sort for efficient multi-passage processing. The system achieved an average NDCG@10 of 50.9 on the BEIR benchmark, outperforming the RankT5 baseline by 1.3 points, and exhibited strong robustness to positional bias and initial input order.
Key insights
- The FiD architecture, by encoding each passage independently with explicit identifiers, effectively mitigates positional bias and the 'Lost in the Middle' problem seen in monolithic LLM inputs.
- Generating passages in increasing order of relevance (least to most relevant) provides a more effective reasoning chain for relevance calibration compared to generating in decreasing order or simple discrimination.
- The m-ary tournament sort algorithm enables efficient scaling of listwise reranking, achieving computational complexity comparable to pointwise methods while leveraging the benefits of comparative passage analysis.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.