Reproduce “Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zailong Tian, Zhuoheng Han, Yanzhe Chen, Haozhe Xu, Xi Yang, Richeng Xuan, Houfeng Wang, Lizi Liao
This research systematically diagnosed pervasive overconfidence in Large Language Models (LLMs) when used as automated judges and introduced the TH-Score, a novel metric for evaluating confidence-accuracy alignment. The study developed LLM-as-a-Fuser, an ensemble framework that synthesizes judgments and critiques from multiple LLMs, achieving an 8.86% accuracy increase and a 5.36% ECE reduction compared to a top-performing standalone model.
Key insights
- The 'Overconfidence Phenomenon' is pervasive among state-of-the-art LLMs, with models frequently overestimating their correctness, as evidenced by high Expected Calibration Error (ECE) values and reliability plots.
- Integrating and analyzing critiques (explanations) from multiple LLM judges, rather than just their final decisions, is key to significantly improving both the accuracy and calibration of automated evaluation systems.
- The TH-Score effectively quantifies confidence-accuracy alignment in practical, high-stakes confidence intervals, offering a more relevant assessment for data filtering and quality assurance than traditional aggregate metrics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.