Reproduce “InternalInspector $I^2$: Robust Confidence Estimation in LLMs through Internal States” on OpenResearch

About this paper

Mohammad Beigi, Ying Shen, Runing Yang, Zihao Lin, Qifan Wang, Ankith Mohan, Jianfeng He, Ming Jin and 2 more

InternalInspector introduces a framework for robustly estimating the factual correctness of a large language model's outputs by analyzing its full spectrum of internal states across all layers. The method achieved up to a 20.4% increase in accuracy and an 8.9% reduction in Expected Calibration Error compared to existing baselines on various NLU and NLG tasks, enhancing confidence estimation and hallucination detection.

Key insights

• A holistic analysis of all internal states (attention, feed-forward, and activation states across all layers) provides a more robust and discriminative signal for estimating LLM output correctness than relying solely on final hidden states.
• Supervised contrastive learning is critical for enabling the framework to learn fine-grained distinctions in internal states that effectively separate correct predictions from incorrect ones in the embedding space.
• Different internal state types contribute distinctly to confidence estimation: Feed-Forward Networks are particularly influential for factual recall, while attention states are more crucial for contextual understanding, with their integration yielding optimal performance.

Summary by alphaXiv.