Reproduce “MR-JEPA: A General Purpose Video Foundation Model for Cardiac MRI” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Athira J. Jacob, Puneet Sharma, Dorin Comaniciu, Daniel Rueckert

MR-JEPA is a compact, fully self-supervised video foundation model that represents cardiac MRI through joint spatial and temporal processing across cine, late gadolinium enhancement, and mapping sequences. Using frozen embeddings for downstream tasks, it achieved an LV ejection-fraction MAE of 4.79 percentage points, improved strain estimation over the evaluated baselines, and reached a disease-classification macro AUC of 0.868, although a larger text-supervised CMR model performed better for classification.

Key insights

  • Representing frames and ordered image slices as a shared spatiotemporal sequence allows one encoder to capture cardiac motion as well as cross-slice anatomical structure.
  • Domain-specific 2D initialization and spatiotemporal masking were important for transfer: removing either caused LGE scar-detection performance to fall to chance level in the reported ablations.
  • More frames did not necessarily improve learning; increasing clips from eight to 16 frames reduced cine-view classification accuracy and made optimization less efficient under the tested setup.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.