Reproduce “KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yushi Sun, Kai Sun, Yifan Ethan Xu, Xiao Yang, Xin Luna Dong, Nan Tang, Lei Chen

KERAG (Knowledge-Enhanced Retrieval-Augmented Generation) introduces a novel KG-based RAG pipeline that retrieves broad subgraphs, intelligently filters information, and uses fine-tuned Chain-of-Thought summarization to answer complex questions. The system achieved a truthfulness score of 0.529 on CRAG, outperforming previous state-of-the-art solutions by 7.1%, and 0.860 on Head2Tail, an improvement of 7-8%, while significantly reducing LLM hallucination.

Key insights

  • Shifting complex reasoning from the retrieval-path generation phase to the post-retrieval summarization phase leverages LLMs' strengths more effectively, allowing for broader knowledge exploration.
  • Iterative, LLM-guided schema-level planning for retrieval enables dynamic determination of relevant neighborhood scope, balancing coverage and managing knowledge overload.
  • Supervised fine-tuning of the CoT summarizer, combined with a self-correction mechanism based on ground truth, significantly enhances the LLM's ability to generate accurate reasoning chains and answers.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.