Reproduce “OmniSat: Self-Supervised Modality Fusion for Earth Observation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Guillaume Astruc, Nicolas Gonthier, Clement Mallet, Loic Landrieu

OmniSat aligns image and time-series observations of the same ground patches across sensors, then uses contrastive learning and masked reconstruction to pretrain without labels. This lets multimodal pretraining improve Sentinel-2-only tree-species classification on TreeSatAI-TS: weighted F1 rises from 49.7 without pretraining to 62.9 with it. The result shows a benefit for this task, not equal gains across datasets or modalities.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.