Reproduce “ViTs for SITS: Vision Transformers for Satellite Image Time Series” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Michail Tarasiou, Erik Chavez, Stefanos Zafeiriou
Imperial College London researchers developed the Temporo-Spatial Vision Transformer (TSViT), a fully-attentional model for satellite image time series data, which enhances crop type recognition and land cover classification. The model achieves state-of-the-art performance, outperforming prior methods by notable margins in both semantic segmentation and object classification on public datasets, while maintaining a comparable computational footprint.
Key insights
- Temporo-spatial factorization (temporal-then-spatial) for SITS processing is significantly more effective than spatio-temporal factorization, yielding a 29.7% absolute mIoU improvement in ablation studies.
- Acquisition-time-specific temporal positional encodings are crucial for handling irregular SITS data, improving performance by 2.8% mIoU compared to fixed encodings.
- Utilizing multiple learnable class tokens for different classes enhances the model's capacity and ability to gather class-specific evidence, leading to a 5.1% mIoU gain.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.