Reproduce “On Exploring the Reasoning Capability of Large Language Models with Knowledge Graphs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pei-Chi Lo, Yi-Hang Tsai, Ee-Peng Lim, San-Yih Hwang
The authors test whether language models can produce multi-hop knowledge-graph paths using a staged process that selects relevant sentences, links entities to DBpedia, and generates paths. For text-davinci-003, multi-step prompting lowered graph-edit distance from 0.37 to 0.29 and formatting errors from 0.46 to 0.35; invalid relations fell from 0.34 to 0.30, with no significant improvement reported. GPT-4 single-step performed better.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.