Reproduce “Cross-Representation Benchmarking in Time-Series Electronic Health Records for Clinical Outcome Prediction” on OpenResearch

About this paper

Tianyi Chen, Mingcheng Zhu, Zhiyao Luo, Tingting Zhu

Researchers at the University of Oxford conducted the first comprehensive benchmark to compare multivariate time-series, event stream, and textual event stream representations of Electronic Health Records for clinical outcome prediction across two diverse datasets. Their findings indicate that event stream models generally achieve the strongest performance, while Large Language Models are competitive in short-term, dense clinical scenarios, and optimal feature selection is dependent on the specific clinical context.

Key insights

• Event stream representations consistently delivered the strongest overall predictive performance across both ICU and longitudinal clinical tasks, highlighting their robustness in handling EHR data's irregular nature.
• Pre-trained event stream models like CLMBR demonstrated superior sample efficiency in few-shot learning settings, while simpler count-based models were competitive or even surpassed them with ample training data.
• Large Language Models proved effective in short-term, dense clinical scenarios (e.g., ICU mortality) but struggled with the long, sparse contexts characteristic of longitudinal EHR data.
• The optimal strategy for feature pruning in multivariate time-series models is context-dependent, with moderate pruning beneficial for ICU tasks and retaining all features crucial for longitudinal care.

Summary by alphaXiv.