Reproduce “Fusing Knowledge and Language: A Comparative Study of Knowledge Graph-Based Question Answering with LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vaibhav Chaudhary, Neha Soni, Narotam Singh, Amita Kapoor
A comparative study analyzed three methodologies (spaCy, Stanford CoreNLP-OpenIE, GraphRAG) for integrating knowledge graph triplets with Large Language Models to enhance question answering. GraphRAG demonstrated superior performance for complex, thematic queries and semantic understanding, while spaCy offered a lightweight, high-precision approach, and CoreNLP OpenIE provided broad factual coverage, each with distinct technical and performance trade-offs.
Key insights
- GraphRAG consistently demonstrated superior reasoning abilities and overall performance for complex, thematic queries, benefiting from its hierarchical knowledge representation and multi-hop reasoning capabilities.
- spaCy provides a lightweight, high-precision baseline for triplet extraction, while CoreNLP OpenIE offers broader factual coverage at the cost of higher computational overhead and potential noise.
- A hybrid pipeline combining spaCy for precision, CoreNLP for expanded recall, and GraphRAG for deep reasoning is recommended to leverage the complementary strengths of these tools effectively.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.