Reproduce “Mitigating Lost-in-Retrieval Problems in Retrieval Augmented Multi-Hop Question Answering” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rongzhi Zhu, Xiangyu Liu, Zequn Sun, Yiwei Wang, Wei Hu

Researchers from Nanjing University and the University of California, Merced developed ChainRAG, a progressive retrieval and rewriting framework to mitigate the "lost-in-retrieval" problem in multi-hop question answering by explicitly addressing ambiguous references in decomposed sub-questions. The framework consistently outperformed existing RAG baselines, achieving up to a 60% increase in F1 score on the MuSiQue dataset and demonstrating robustness across various LLMs.

Key insights

  • The "lost-in-retrieval" problem significantly degrades multi-hop QA performance when decomposed sub-questions lack clear entities, as evidenced by an average 18.29% drop in Recall@2 for second sub-questions.
  • Explicitly rewriting ambiguous sub-questions with concrete entities derived from preceding answers is critical for improving retrieval accuracy and maintaining the reasoning flow in multi-hop QA.
  • A fine-grained sentence graph, combining entity, semantic, and structural connections, serves as an effective knowledge organization and retrieval mechanism for complex multi-hop questions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.