Reproduce “Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output” on OpenResearch

About this paper

Hithesh Sankararaman, Mohammed Nasheed Yasin, Tanner Sorensen, Alessandro Di Bari, Andreas Stolcke

Uniphore Software Systems developed Provenance, a light-weight, cross-encoder-based fact-checker for Retrieval-Augmented Generation (RAG) LLM outputs, which achieved competitive hallucination detection performance with significantly smaller models. It demonstrated high Area Under the Receiver Operating Characteristic Curve (AUC) scores across various datasets and surpassed larger LLMs in accuracy on benchmarks like HaluEval and HaluBench.

Key insights

• Compact cross-encoder models, significantly smaller than general-purpose LLMs, can achieve highly effective and reliable hallucination detection in RAG systems.
• Explicitly factoring the original query into both context selection and the NLI-based fact-checking process leads to more context-aware and accurate assessments.
• Careful selection and weighting of context items using strategies like TopP, combined with aggregation functions like 'max', enhance performance, especially in multi-hop reasoning scenarios.

Summary by alphaXiv.