Reproduce “HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pranoy Panda, Ankush Agarwal, Chaitanya Devaguptapu, Manohar Kaul, Prathosh A P
HOLMES constructs query-focused, context-aware, hyper-relational knowledge graphs as the sole input for Large Language Models to answer multi-hop questions. The approach achieves up to 22% Exact Match improvement on HotpotQA compared to base LLM and reduces input token count by up to 67% over prior state-of-the-art methods.
Key insights
- Distilling query-relevant, context-aware knowledge into a hyper-relational graph as the exclusive LLM input significantly enhances multi-hop reasoning accuracy and efficiency.
- Dynamically generating a query-aligned knowledge schema, complemented by an auxiliary schema derived from unannotated questions, effectively prunes irrelevant information, substantially reducing LLM input overhead.
- Explicitly incorporating context (e.g., source document title) as an attribute within knowledge graph triples addresses ambiguity and improves the reliability of LLM inferences.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.