Reproduce “Language Models Prefer What They Know: Relative Confidence Estimation via Confidence Preferences” on OpenResearch

About this paper

Vaishnavi Shrivastava, Ananya Kumar, Percy Liang

Researchers at Stanford University developed a relative confidence estimation method for large language models, where models make pairwise comparisons of their confidence in answering questions. This approach, which aggregates these preferences using rank aggregation algorithms, consistently improved selective classification AUC by 1.7% to 3.5% over existing absolute confidence methods across various state-of-the-art LLMs and datasets, enhancing the reliability of uncertainty quantification, especially for closed-source models.

Key insights

• LLMs, like humans, find it easier and more robust to make relative comparative judgments of confidence rather than isolated absolute assessments.
• Access to the model's own generated answer is crucial for highly reliable confidence judgments, indicating that self-assessment relies heavily on the output itself.
• Incorporating Chain-of-Thought (CoT) reasoning into relative confidence prompts did not improve performance and sometimes slightly decreased it, suggesting that explicit reasoning for self-confidence might not be beneficial in this context.

Summary by alphaXiv.