Reproduce “Miscalibrated AI improves human calibration: An alignment problem for AI-assisted Decision education” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Phillip C. Hegeman, Eeshan Hasan, Eungik Lee, Andrew Caplin, Jennifer S. Trueblood

The authors test whether AI confidence helps novices judge blood cells without AI. Three hundred participants practiced with the same approximately 94%-accurate model, shown with conservative probabilities, recalibrated probabilities, or labels without confidence. The conservative display reduced overconfidence and improved calibration more than the other displays, without a condition-specific gain in accuracy or discrimination, in this single-session task.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.