Reproduce “Hx2Dx: From Anamnesis to Diagnosis - A Multimodal Arena for Conversational Clinical Reasoning in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Federico Ravenda, Mirko Cesarini, Daniele Montagnani, Volodymyr Karpenko, Antonietta Mira, Andrea Raballo
Hx2Dx ARENA compares models diagnosing matched NEJM cases from completed information or through simulated encounters. In encounters, models must request history, examinations, and tests from case sources before giving a diagnosis. All seven models lose accuracy interactively; GPT Luna 6 falls from 83.9% to 47.4%. A stronger interviewer’s transcript improves weaker models, but evaluation on educational cases does not establish clinical readiness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.