Reproduce “A Survey on the Robustness of Computer Vision Models against Common Corruptions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shunxin Wang, Raymond Veldhuis, Christoph Brune, Nicola Strisciuglio
The authors provide a comprehensive survey and unified benchmark, `CorruptionBenchCV`, to evaluate the robustness of computer vision models against common corruptions. They find that while Transformer-based models generally outperform CNNs, the gains in corruption robustness from increasing model size or pre-training data diminish significantly beyond a certain threshold, challenging the prevailing scaling paradigm.
Key insights
- Corruption robustness does not scale linearly with model size or pre-training data beyond approximately 100 million parameters; larger models offer diminishing returns in robustness.
- Many models achieving high robustness, particularly superior Transformer-based architectures, often exhibit higher Expected Calibration Error (ECE), indicating less reliable prediction confidence.
- Knowledge distillation can be a more effective strategy for leveraging large-scale pre-training data to improve corruption robustness compared to simply fine-tuning larger models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.