Reproduce “Evaluating and Calibrating LLM Confidence on Questions with Multiple Correct Answers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuhan Wang, Shiyu Ni, Zhikai Ding, Zihang Zhan, Yuanzi Li, Keping Bi
The paper addresses the miscalibration of large language model confidence when questions have multiple correct answers, where current methods mistake semantic diversity for uncertainty. It introduces the MACE benchmark and the Semantic Confidence Aggregation (SCA) method, which improves calibration performance for such questions, achieving state-of-the-art AUROC on mixed-answer settings.
Key insights
- LLM accuracy consistently increases with a greater number of correct answers for a question, while estimated confidence from existing methods paradoxically decreases, indicating severe miscalibration.
- Larger LLMs (e.g., LLaMA-3.1-70B) exhibit more pronounced confidence degradation in multi-answer settings due to their broader knowledge leading to a wider range of diverse, yet correct, answers.
- The calibration performance (AUROC) of most existing methods degrades significantly when evaluated on question sets with mixed ground-truth answer cardinalities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.