Reproduce “Human-AI Teaming Through the Lens of Calibration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eric Nalisnick, Chi Zhang, Sophia Qian, Yixin Wang
The authors analyze how calibrated human and model predictions can be combined or routed, using calibration to track whether each teammate’s information is preserved. They show combination can retain model calibration yet lose human calibration, while optimal delegation requires a rejector fine-grained enough to identify when the human is better. Delegation is theoretical; hidden information can leave excess risk under stated assumptions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.