Reproduce “Decision and System One models in rheumatology: accuracy and calibration on specialty-training examination questions and diagnostic case vignettes” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alfredo Madrid-García, Diego Benavent, Fabian Lechner, Phillip Kremer, Johannes Knitza, Beatriz Merino-Barbancho
The authors evaluate fixed-choice clinical decision models that select from predefined answers and return probabilities, using rheumatology examination questions and diagnostic vignettes. Jev 1.13 scored 93.0% on Spanish examination questions, but 50.0% on 52 vignettes where the reference diagnosis was offered, compared with GPT-6 Sol’s 76.9%. These benchmark results do not establish performance in routine care.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.