Reproduce “Dissecting Agentic RAG: A Component Ablation for Multi-Hop QA with a Local 7B Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sheroz Shaikh
An ablation study systematically evaluates components of an agentic Retrieval-Augmented Generation (RAG) pipeline for multi-hop question answering, utilizing a local 7B language model. This work identifies iterative evidence accumulation as the primary performance driver, improving Exact Match by 10.1% and F1 by 7.6% over a single-pass baseline, and shows that fixed hybrid retrieval can outperform adaptive routing.
Key insights
- Iterative evidence accumulation is the primary driver of performance gains in agentic RAG for multi-hop QA, with reducing the loop to a single step resulting in the largest performance drop (-7.1% Exact Match).
- Simpler, fixed hybrid retrieval (Reciprocal Rank Fusion) can outperform a complex rule-based adaptive router, suggesting that adaptive heuristics may sometimes misroute queries and lose complementary retrieval signals.
- For typical multi-hop questions, 2-3 retrieval iterations are largely sufficient, capturing about 95% of the performance gains from deeper loops, which offers a balance between accuracy and computational cost.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.