Reproduce “Your Model is Overconfident, and Other Lies We Tell Ourselves” on OpenResearch

About this paper

Timothee Mickus, Aman Sinha, Raúl Vázquez

Researchers at the University of Helsinki and Université de Lorraine empirically demonstrate that commonly used model-based indicators of data difficulty and uncertainty show low agreement with human perceptions of difficulty, often conflating model success and failure. The study highlights that reference-free metrics form an "inverted U-shape," treating genuinely ambiguous examples and confidently incorrect predictions similarly.

Key insights

• Human-based and model-based indicators of data difficulty show consistently low Spearman correlations, often below "rho"=0.25 for MNLI and at most ",rho"=0.44 for SNLI.
• Reference-free model-based indicators (e.g., model confidence, prediction set size) conflate model success and failure, often forming an 'inverted U-shape' where high uncertainty is assigned to both genuinely ambiguous examples and confidently incorrect predictions.
• Training with 'soft' (continuous human label distributions) targets showed some improvements for certain reference-free indicators but did not fully align model-based metrics with human perceptions of difficulty.

Summary by alphaXiv.