Reproduce “Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chuangtao Ma, Sriom Chakrabarti, Arijit Khan, Bálint Molnár
KG-RAG4SM is a model that enhances schema matching by integrating external knowledge graphs with large language models through retrieval-augmented generation. It consistently outperforms existing LLM and PLM baselines in Precision and F1-score, achieving up to 150% higher Precision on the Synthea dataset, and effectively mitigates LLM hallucination in complex, domain-specific contexts without requiring model re-training.
Key insights
- Augmenting LLMs with carefully retrieved and refined subgraphs from external KGs effectively mitigates LLM hallucination and improves semantic matching accuracy in complex, domain-specific scenarios.
- Smaller, instruction-tuned LLMs can perform better within KG-RAG frameworks than larger, general-purpose LLMs, possibly due to fewer internal knowledge conflicts with augmented external data.
- Providing relevant KG subgraphs can reduce LLM inference time by helping the model locate knowledge more efficiently, leading to faster generation times compared to unaugmented LLMs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.