Reproduce “Tokenization Tradeoffs in Structured EHR Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lin Lawrence Guo, Santiago Eduardo Arciniegas, Joseph Jihyung Lee, Adam Paul Yan, George Tomlinson, Jason Fries, Lillian Sung
Research by The Hospital for Sick Children and Stanford University systematically investigated tokenization strategies for structured electronic health record foundation models, revealing that joint event encoding and positional time encoding enhance predictive performance and computational efficiency. The benefits of joint event encoding, specifically, transferred to an external adult intensive care unit cohort despite high out-of-vocabulary rates.
Key insights
- Systematically encoding clinically meaningful structure directly into tokens, such as through joint event encoding, improves predictive performance and reduces computational costs in EHR foundation models.
- The 'local binding problem' is a critical factor, where joint encoding mitigates the model's burden to learn complex adjacent token relationships (e.g., code-attribute pairs) from limited data.
- While explicit temporal and workflow encodings offer some localized benefits, their generalizability is more context-dependent, with only the event encoding advantage showing robust transfer across different institutions and patient populations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.