Reproduce “ReMedi: Reasoner for Medical Clinical Prediction” on OpenResearch

About this paper

Yushi Cao, Yiming Chen, Hongchao Jiang, Hung-yi Lee, Robby T. Tan

ReMedi is a framework that improves clinical outcome prediction from electronic health records by enhancing Large Language Models' (LLMs) intrinsic reasoning capabilities through self-generated rationale-answer pairs and preference optimization. It achieved an F1 score of 97.6% for mortality prediction, 90.4% for readmission prediction, and 55.5% for length of stay, outperforming state-of-the-art baselines.

Key insights

• Explicitly enhancing LLMs' internal reasoning with self-generated rationale-answer pairs and preference optimization significantly improves clinical prediction accuracy over external knowledge retrieval or general self-improvement methods.
• Targeting and re-generating data from "challenging samples" where the model initially struggles is critical for robust performance in complex clinical prediction tasks.
• Optimizing for rationale-answer coherence through DPO leads to better alignment between an LLM's generated reasoning and its final prediction, enhancing interpretability and trustworthiness.

Summary by alphaXiv.