Reproduce “EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiyoun Kim, Muhan Yeo, Eunhye Jang, Jeewon Yang, Hangyul Yoon, Su Ji Lee, Hee Jo Han, Hee-Jae Jung and 9 more
EHRNote-ChatQA tests whether models can answer clinical questions across a patient’s discharge summaries and identify the notes and sections supporting each answer. It pairs content questions with source-location questions in multi-turn sequences, using expert-reviewed, de-identified records from one hospital system. Across 24 models, source accuracy averaged 22.0 percentage points below content accuracy. This does not establish clinical readiness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.