Reproduce “KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiho Kim, Yeonsu Kwon, Yohan Jo, Edward Choi

KG-GPT is a three-stage framework developed at KAIST and Seoul National University that enables large language models to reason over knowledge graphs using in-context learning. This framework achieves competitive performance in fact verification and multi-hop question answering tasks, often outperforming baseline and some fully-supervised models in few-shot settings.

Key insights

  • Effective decomposition of complex, multi-hop reasoning into simpler segments through Sentence Segmentation is crucial but also the primary bottleneck, especially for complex queries.
  • The Graph Retrieval mechanism demonstrates high robustness in identifying relevant relations and triples for evidence graphs, given accurate initial segmentation.
  • The number of in-context examples required for competitive performance varies with task complexity, with simpler tasks showing robustness even with fewer shots.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.