Reproduce “Uncertainty evaluation of segmentation models for Earth observation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Melanie Rey, Andriy Mnih, Maxim Neumann, Matt Overlan, Drew Purves
The authors benchmark uncertainty estimates for satellite image segmentation by testing pixel-error detection, image-level quality assessment, post-hoc rejection, and artificial corruption detection. They find uncertainty is more informative at the image level than for locating individual errors, while results vary by architecture, dataset, task, and noise levels. Findings cover PASTIS and ForTy, not real-world corruption broadly.
Key insights
- Transformer architectures like TSViT demonstrated superior capability in identifying segmentation errors compared to convolutional models.
- Uncertainty metrics show differential utility: MaxProb is more effective for identifying misclassified pixels, while Entropy performs better for detecting noisy inputs.
- Stochastic Segmentation Networks (SSNs) offer an efficient alternative to ensembles, providing a strong balance between predictive performance, model efficiency, and robustness to input noise, particularly benefiting from noise training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.