Reproduce “CardioMeta: Calibrated Multi-Task Prediction of Diabetes, Hypertension, and Cardiovascular Disease Across Population and EHR Data” on OpenResearch

About this paper

S M Asif Hossain, Ruksat Khan Shayoni, M. F. Mridha, Jungpil Shin

CardioMeta, a multi-task learning framework, predicts diabetes, hypertension, and cardiovascular disease status using population survey and EHR data while addressing label leakage and probability calibration. It achieves competitive discrimination (macro-AUROC of 0.839) with superior calibration (ECE of 0.024) in leakage-reduced settings, demonstrating the impact of domain shift and the value of localized fine-tuning.

Key insights

• Direct inclusion of label-defining features (full-clinical setting) leads to substantial performance inflation (e.g., AUROC increase from 0.839 to 0.899), highlighting the critical importance of leakage-reduced evaluation for clinically meaningful risk prediction.
• While multi-task learning provides modest discrimination gains, it offers a structured approach to learn shared representations for interconnected conditions, with calibration consistently improving the reliability of predictive probabilities.
• Models trained on population survey data (NHANES) exhibit significant performance degradation when directly transferred to electronic health record data (MIMIC-IV), underscoring challenges of domain shift, though performance largely recovers with limited fine-tuning.

Summary by alphaXiv.