Reproduce “LLM Internal States Reveal Hallucination Risk Faced With a Query” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ziwei Ji, Delong Chen, Etsuko Ishii, Samuel Cahyawijaya, Yejin Bang, Bryan Wilie, Pascale Fung

Researchers from CAiRE at HKUST investigated whether Large Language Models (LLMs) can estimate their own hallucination risk for a query by analyzing internal states before response generation. Their internal state-based estimator achieved an average accuracy of 84.32% in predicting hallucination across 15 diverse Natural Language Generation tasks, operating with high efficiency at 0.05 seconds per sample.

Key insights

  • LLMs possess an inherent ability to recognize whether a query's underlying information was part of their training data, discernible through their internal states.
  • Deeper layers within LLMs are more effective at encoding and revealing patterns indicative of hallucination risk compared to earlier layers, suggesting a representation of higher-level uncertainty.
  • An internal state-based mechanism can serve as a proactive 'early warning system' for potential hallucination, allowing for interventions like Retrieval Augmented Generation (RAG) before unfaithful content is produced.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.