Reproduce “Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shuai Ma, Ying Lei, Xinru Wang, Chengbo Zheng, Chuhan Shi, Ming Yin, Xiaojuan Ma

A framework was developed for AI-assisted decision-making that estimates and leverages both human and AI correctness likelihood to foster appropriate trust. This approach led to enhanced human-AI team performance and better trust calibration, outperforming methods relying solely on AI confidence.

Key insights

  • It is feasible to accurately estimate a human's task-instance-specific correctness likelihood, which positively correlates with their actual performance.
  • Incorporating both human and AI correctness likelihoods provides a more complete and effective basis for calibrating human trust in AI systems.
  • Adaptive interaction strategies, designed based on cognitive principles, can effectively reduce over-trust in AI and promote more appropriate reliance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.