Reproduce “From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Meftun Akarsu, Recep Kaan Karaman, Christopher Mierbach
This study benchmarks ten retrieval strategies for Retrieval-Augmented Generation (RAG) on financial documents containing both text and tables, utilizing the T²-RAGBench dataset. It demonstrates that a two-stage pipeline combining hybrid retrieval with neural reranking achieves superior performance, reaching a Recall@5 of 0.816 and MRR@3 of 0.605, while also revealing that sparse lexical retrieval often outperforms dense methods in this specific domain.
Key insights
- A two-stage retrieval pipeline, combining hybrid retrieval with neural cross-encoder reranking, is crucial for achieving high performance in RAG systems operating on text-and-table documents.
- Sparse lexical retrieval (BM25) frequently outperforms state-of-the-art dense semantic retrieval for financial documents, suggesting the importance of precise term matching in specific domains.
- Query expansion techniques like HyDE and Multi-Query Retrieval provided negligible or negative benefits for precise numerical QA, while contextual retrieval consistently offered moderate improvements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.