Reproduce “HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses” on OpenResearch

About this paper

Xinke Jiang, Ruizhe Zhang, Yongxin Xu, Rihong Qiu, Yue Fang, Zhiyuan Wang, Jinyi Tang, Hongxin Ding and 3 more

The HyKGE framework enhances the accuracy and reliability of medical LLM responses by integrating a novel Retrieval-Augmented Generation approach. It leverages an LLM to generate 'Hypothesis Outputs' for enriched query understanding before Knowledge Graph retrieval and employs a fragment-granularity aware reranking module, outperforming baselines by up to 28.15% in exact match scores on medical Q&A datasets and significantly reducing hallucinations.

Key insights

• Leveraging an LLM's zero-shot reasoning for 'hypothesizing' potential answers before KG retrieval effectively compensates for sparse user queries and guides more precise knowledge exploration.
• A fine-grained, fragment-aware reranking strategy (using both user query and LLM-generated hypothesis fragments) is crucial for balancing the diversity and relevance of retrieved knowledge, preventing redundancy while maintaining coverage.
• Structured reasoning chains from Knowledge Graphs, when effectively pruned and presented, can correct LLM hallucinations and significantly improve the factual accuracy and interpretability of medical LLM responses.

Summary by alphaXiv.