Reproduce “Metacognition and Uncertainty Communication in Humans and Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mark Steyvers, Megan A.K. Peters

This conceptual review examines the metacognitive capabilities of Large Language Models (LLMs) in comparison to humans, synthesizing existing research on how LLMs monitor and communicate uncertainty. The paper highlights both convergences, such as similar behavioral patterns and consistency-based confidence generation, and divergences, including LLMs' overconfidence and reluctance to verbally express uncertainty, which has implications for human-AI collaboration.

Key insights

  • Implicit measures of LLM confidence often correlate more strongly with task accuracy than explicit, verbally expressed confidence, suggesting a gap between internal uncertainty representation and linguistic communication.
  • Both humans and LLMs exhibit overconfidence, and LLMs, similar to humans, can leverage consistency across internal alternatives to generate confidence judgments.
  • LLM metacognitive abilities, particularly calibration, are trainable through finetuning, but their sensitivity may remain domain-specific, unlike potentially more stable human metacognition.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.