Reproduce “Calibrating LLM Confidence by Probing Perturbed Representation Stability” on OpenResearch

About this paper

Reza Khanmohammadi, Erfan Miahi, Mehrsa Mardikoraem, Simerjot Kaur, Ivan Brugere, Charese H. Smiley, Kundan Thind, Mohammad M. Ghassemi

A new method, Calibrating LLM Confidence by Probing Perturbed Representation Stability (CCPS), estimates Large Language Model confidence by assessing the robustness of internal representations to targeted adversarial perturbations. This approach consistently enhances both calibration and discriminative power across diverse LLM architectures and tasks, while maintaining high computational efficiency.

Key insights

• An LLM's actual confidence in an answer correlates with the stability and robustness of its internal representations under subtle adversarial perturbations.
• Probing the dynamic reaction of hidden states to targeted internal stress yields a more accurate and holistic signal for confidence than static internal states or direct model self-assessment.
• Perturbation-derived features are the primary contributors to achieving superior performance across both calibration and discriminative metrics, highlighting their distinct informative value.

Summary by alphaXiv.