Reproduce “Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jinyi Han, Tingyun Li, Shisong Chen, Jie Shi, Xinyi Wang, Guanglei Yue, Jiaqing Liang, Xin Lin and 3 more

Researchers introduce FineCE, a method for providing accurate, fine-grained confidence scores to Large Language Models throughout the text generation process, enhancing trustworthiness and allowing for dynamic self-correction. The approach achieves significant improvements in calibration and discrimination over baselines, leading to up to a 39.5% increase in accuracy when used for confidence-based filtering on datasets like GSM8K.

Key insights

  • Fine-grained confidence estimation is most effectively learned through Instruction Fine-Tuning, leveraging the LLM's natural language generation capabilities rather than a separate classification head.
  • Integrating 'backward confidence' from future generation paths (BCI) during inference significantly improves the calibration and accuracy of confidence estimates.
  • Reliable confidence signals can be obtained surprisingly early in the generation process (~30% of tokens), enabling proactive interventions and early termination of potentially incorrect outputs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.