Reproduce “CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anthony Fuller, Koreen Millard, James R. Green
CROMA is a self-supervised learning framework designed for remote sensing that learns rich, multimodal representations by combining contrastive learning and masked autoencoding on spatially aligned radar and optical satellite imagery. The framework introduces novel relative position encoding strategies, enabling it to consistently outperform existing methods across multiple classification and segmentation benchmarks while also generalizing to larger geographical areas.
Key insights
- Combining contrastive learning with masked autoencoding creates robust representations that capture both structural semantic relationships and detailed reconstruction capabilities from multimodal data.
- Novel relative position encoding strategies, X-ALiBi and 2D-ALiBi, enable Vision Transformers to extrapolate effectively to larger image sizes and improve cross-modal fusion.
- Leveraging spatially aligned radar (Sentinel-1) and optical (Sentinel-2) data provides complementary information, enhancing the learned representations for Earth observation tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.