Reproduce “Uncalibrated Reasoning: GRPO Induces Overconfidence for Stochastic Outcomes” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Michael Bereket, Jure Leskovec

The authors test whether models can learn calibrated probabilities from observed binary outcomes on synthetic and CRISPR tasks. They compare PPO, RLOO, GRPO, and GRPO without standard deviation normalization, using log-likelihood rewards. On held-out CRISPR perturbations, GRPO has ECE 0.292 versus 0.036 without standard normalization, while thresholded accuracy is similar. This does not establish rigorous uncertainty reasoning in generated explanations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.