Reproduce “Bias patterns in the application of LLMs for clinical decision support: A comprehensive study” on OpenResearch

About this paper

Raphael Poulain, Hamed Fayyaz, Rahmatollah Beheshti

Researchers at the University of Delaware conducted a comprehensive study on bias patterns in Large Language Models (LLMs) used for clinical decision support, evaluating eight models across diverse clinical tasks and prompting strategies. The study found that biases are prevalent and complex, with model size and domain-specific fine-tuning not guaranteeing fairness, and demonstrated that Chain of Thought prompting can generally reduce bias and improve preferred outcomes.

Key insights

• Bias is prevalent and complex in LLMs for clinical decision support, varying significantly across different models, tasks, and demographic groups, challenging assumptions about uniform fairness.
• Neither larger model size nor domain-specific fine-tuning consistently reduced bias; some clinically fine-tuned models exhibited higher disparities than general-purpose ones, which is a critical finding for medical AI development.
• Chain of Thought (CoT) prompting generally serves as an effective bias mitigation strategy by encouraging LLMs to articulate their reasoning steps, leading to reduced disparities and improved preferred clinical outcomes in most cases.

Summary by alphaXiv.