Reproduce “Cycles of Thought: Measuring LLM Confidence through Stable Explanations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Evan Becker, Stefano Soatto
A framework called "stable explanations confidence" quantifies uncertainty in large language models by assessing the stability and logical consistency of their generated explanations. This approach, developed by Evan Becker and Stefano Soatto from UCLA, consistently outperformed baseline methods in discriminating correct from incorrect answers and improved confidence score calibration across multiple question-answering datasets.
Key insights
- The stability and logical consistency of LLM-generated explanations serve as a reliable proxy for the model's confidence in its answers.
- Explanations supporting correct answers tend to exhibit greater stability and logical consistency with the question compared to explanations for incorrect answers.
- Reweighting explanations by their entailment probabilities (how well they follow from the question) is crucial for improving the reliability of confidence scores.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.