Reproduce “CompactRAG: Reducing LLM Calls and Token Overhead in Multi-Hop Question Answering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hao Yang, Zhiyu Yang, Xupeng Zhang, Wei Wei, Yunjie Zhang, Lin Yang
CompactRAG introduces a framework for Multi-Hop Question Answering that significantly reduces Large Language Model calls and token consumption by decoupling offline corpus processing from online reasoning. It achieves competitive accuracy on benchmark datasets while fixing LLM usage to two calls per query and lowering token overhead by up to 81%.
Key insights
- Decoupling the extensive corpus processing into a one-time offline step allows for a highly efficient online inference process with minimal, fixed LLM calls.
- Utilizing a compact 'atomic QA knowledge base' paired with specialized lightweight modules effectively replaces the need for repeated, expensive LLM-based reasoning in intermediate hops.
- Explicitly re-grounding entities in subsequent sub-questions through a rewriter module is essential to prevent semantic drift and maintain factual consistency during multi-hop reasoning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.