Reproduce “Reinforced Information Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chaofan Li, Zheng Liu, Jianlyv Chen, Defu Lian, Yingxia Shao
Reinforced Information Retrieval introduces a self-boosting framework that jointly adapts a pre-trained retriever and an LLM-based generator using unlabeled target-domain data. This approach, leveraging mutual reinforcement learning, achieved substantial performance gains, increasing nDCG@10 by an average of 16.9% on BEIR and 12.6% on AIR-Bench datasets for a base Contriever model, while consistently outperforming existing domain adaptation methods.
Key insights
- Jointly adapting both the retriever and an LLM-based generator through mutual reinforcement learning is more effective for cross-domain retrieval than optimizing them in isolation or using static generators.
- Fine-grained feedback mechanisms, specifically DPO for generator training and knowledge distillation from the LLM's re-ranker for retriever training, are critical for capturing nuanced relevance signals and enhancing domain adaptation.
- The proposed 'proximity objective' simplifies retriever learning, particularly for question-answering tasks, by focusing on aligning raw queries with both relevant documents and generated hypothetical documents.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.