Reproduce “MedRiskEval: Medical Risk Evaluation Benchmark of Language Models, On the Importance of User Perspectives in Healthcare Settings” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jean-Philippe Corbeil, Minseon Kim, Maxime Griot, Sheela Agarwal, Alessandro Sordoni, Francois Beaulieu, Paul Vozila
The authors introduce MedRiskEval, an evaluation of medical AI risks from patient, clinician, and general-user perspectives. Its PatientSafetyBench contains 466 synthetic, short, single-turn patient queries across five risk categories, scored on a five-level safety rubric. Among displayed open-weight models, mean misdiagnosis and overconfidence scores are about 2.3–2.8, near unsafeguarded engagement; category averages do not establish harm from individual answers.
Key insights
- General safety properties of LLMs do not reliably transfer to medical contexts, necessitating domain-specific and multi-faceted safety evaluations.
- Medically adapted LLMs can significantly reduce harmfulness in knowledge-intensive patient safety policies (e.g., medical advice, misinformation) compared to general models.
- Clinical alignment and biomedical pre-training enhance model groundedness and can address behavior-focused patient safety policies, while maintaining general safety requires careful consideration, especially for complex jailbreaking scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.