Reproduce “Winning Solution For Meta KDD Cup' 24” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yikuan Xia, Jiazun Chen, Jun Gao
Peking University's db3 team developed a comprehensive RAG framework that secured first place in all three tasks of the Meta KDD Cup '24 CRAG Challenge. Their solution effectively combines web retrieval and knowledge graph integration, utilizing a fine-tuned Llama-3-8B-instruct model to significantly reduce hallucination and improve answer accuracy.
Key insights
- Meticulous LLM fine-tuning, particularly for teaching the model to identify "invalid questions" and admit "i don't know," is critical for reducing hallucination and improving trustworthiness.
- Integrating structured knowledge graph data, especially with a simplified API interaction method, significantly enhances performance for real-time and fast-changing facts.
- A multi-path RAG system that combines robust web retrieval with strategic prioritization of high-fidelity structured data sources (like KGs) is highly effective in complex scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.