Reproduce “Query-Level Uncertainty in Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Lihu Chen, Gerard de Melo, Fabian M. Suchanek, Gaël Varoquaux

Internal Confidence asks a language model whether it can answer a query accurately before generating an answer. It combines the model’s Yes probability from multiple layers and query-token positions in a training-free, single forward pass. On zero-shot Qwen-14B, it reaches mean AUROC 67.1, but MSP is higher on GSM8K and requires internal-state access.

Key insights

  • Large Language Models can internally self-assess their capability to answer a query by providing a probability for a simple binary "Yes/No" prompt.
  • Rich latent knowledge pertaining to an LLM's uncertainty is distributed across its internal hidden states, spanning multiple layers and token positions.
  • Aggregating these internal signals through a weighted sum, using an attenuated encoding, effectively quantifies query-level uncertainty without explicit training.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.