Reproduce “Grounding LLM Reasoning with Knowledge Graphs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Alfonso Amayuelas, Joy Sain, Simerjot Kaur, Charese Smiley

A framework was introduced to ground Large Language Model (LLM) reasoning with Knowledge Graphs, achieving state-of-the-art performance on domain-specific question-answering tasks. This method improved LLM accuracy and traceability by explicitly linking reasoning steps to structured knowledge, showing over 26.5% gain in Rouge-L scores on GRBench.

Key insights

  • Systematically grounding LLM reasoning in Knowledge Graphs significantly enhances accuracy and interpretability for complex, domain-specific question-answering tasks.
  • The Tree-of-Thought (ToT) reasoning strategy, which explores multiple candidate reasoning paths, consistently outperforms linear (CoT) and graph-merging (GoT) strategies when integrated with KGs.
  • Explicit agent-based interaction with KGs proves more effective for refining answers over longer, more complex reasoning sequences, while automatic graph exploration offers strong performance for shallower tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.