Reproduce “Sequence Composition in EHR Foundation Model Pretraining” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Joshua Placidi, Inigo Stibbe, Alfred J. Balston, Marek Rei, A. Aldo Faisal

The authors introduce OGS, which changes how patient histories are weighted and sampled into pretraining windows for EHR models. Unlike a masked global-stream control, OGS changes exposure and window sampling while blocking cross-patient attention. Across four MIMIC-IV v2.2 tasks, zero-shot macro F1 averaged 0.459 ± 0.017 versus 0.430 ± 0.015 over five seeds; results are limited to one architecture and recipe.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.