Reproduce “Internal Representations as Indicators of Hallucinations in Agent Tool Selection” on OpenResearch

About this paper

Kait Healy, Bharathi Srinivasan, Visakh Madathil, Jing Wu

Amazon researchers introduced a real-time framework that detects tool-calling hallucinations in large language models by analyzing their internal representations during a single forward pass. This method achieved competitive accuracy and notably higher recall for hallucination detection on models like GPT-OSS-20B and Llama-3.1-8B, significantly reducing the computational overhead compared to multi-pass baseline techniques.

Key insights

• The internal representations of LLMs, specifically the hidden states from the final transformer layer, contain sufficient discriminative information to detect tool-calling hallucinations.
• Real-time hallucination detection is feasible and computationally efficient by leveraging the single forward pass of an LLM, avoiding the overhead of multiple generations.
• Simple feature aggregation methods, such as mean pooling across last-layer representations, provide robust and effective input for the hallucination classifier without requiring complex transformations.

Summary by alphaXiv.