Reproduce “Causal Evidence that Language Models use Confidence to Drive Behavior” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dharshan Kumaran, Nathaniel Daw, Simon Osindero, Petar Veličković, Viorica Patraucean
The authors test whether confidence-related signals guide abstention on factual multiple-choice questions. GPT-4o’s calibrated confidence predicted whether it withheld an answer, and activation steering in Gemma 3 27B changed abstention on 500 held-out questions: rates ranged from 66.5% to 7.0%. Increased coverage came with lower accuracy among answered questions; causal evidence is limited to this model and task.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.