Reproduce “Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models” on OpenResearch

About this paper

Wataru Hashimoto, Hidetaka Kamigaito, Taro Watanabe

An empirical study shows that the reliability of large-language-model uncertainty estimates depends strongly on the decoding strategy used to generate responses. Across question answering, summarization, translation, and code generation, Contrastive Search most consistently improves prediction–rejection performance by reducing repetition, while beam-based, factuality-oriented, and contrastive methods vary substantially across models; post-training with RLHF or DPO also changes which decoding strategy is best suited for uncertainty estimation.

Key insights

• Contrastive Search is the most consistently effective decoding strategy for uncertainty estimation among the evaluated aligned models, plausibly because its repetition penalty produces more diverse and less degenerate outputs.
• Beam search can distort probability-based uncertainty, and Diverse Beam Search occasionally yields rankings worse than an uninformative baseline; factuality-oriented methods do not automatically improve confidence calibration.
• The preferred decoding strategy is model- and task-dependent: post-training can change uncertainty behavior, with beam search performing relatively better for SFT models and often degrading after RLHF, while DPO shows fewer improvements over SFT.

Summary by alphaXiv.