Reproduce “Teaching Models to Express Their Uncertainty in Words” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Stephanie Lin, Jacob Hilton, Owain Evans
This work demonstrates that large language models like GPT-3 can be finetuned to express calibrated uncertainty about their answers in natural language. The research evaluates this capability using a new arithmetic benchmark, CalibratedMath, showing that the models maintain calibration under various distribution shifts and leverage pre-existing internal representations of uncertainty.
Key insights
- Large language models like GPT-3 possess latent representations within their pre-trained embeddings that correlate with the correctness of their answers, suggesting an implicit internal understanding of their own uncertainty.
- Finetuning for verbalized probability primarily "elicits" and provides an interface for these pre-existing uncertainty signals rather than teaching the concept from scratch, evidenced by strong performance with few-shot examples.
- Verbalized probability offers a more effective and human-interpretable form of uncertainty communication than traditional logit-based methods, particularly when a model's accuracy varies significantly across different task distributions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.