Reproduce “One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zilin Jing, Vincent Jeanselme, Yuta Kobayashi, Simon A. Lee, Chao Pang, Aparajita Kashyap, Yanwei Li, Xinzhuo Jiang and 1 more

Researchers from Columbia University and UCLA introduced ORA, a marked time-to-event pretraining objective that jointly models event timing and associated measurements in structured Electronic Health Record data, achieving improved and more balanced performance across various clinical prediction tasks compared to existing methods.

Key insights

  • Jointly modeling both event timing and associated numerical values in EHR data, rather than treating them separately or partially, leads to more expressive and generalizable patient representations.
  • The choice of pretraining objective is a critical design consideration for EHR Foundation Models, as ORA's method significantly improves upon next-token prediction and time-to-event-only losses.
  • The benefits of the ORA pretraining objective are backbone-agnostic, demonstrating consistent performance gains when applied to both Transformer and Mamba architectures.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.