Reproduce “COCO-O: A Benchmark for Object Detectors under Natural Distribution Shifts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaofeng Mao, Yuefeng Chen, Yao Zhu, Da Chen, Hang Su, Rong Zhang, Hui Xue
Researchers from Alibaba Group, Zhejiang University, University of Bath, and Tsinghua University introduced COCO-O, a large-scale benchmark for evaluating object detectors against natural distribution shifts, revealing significant performance degradation for standard models and identifying backbone architecture and large-scale pre-training as key factors for improved robustness.
Key insights
- The backbone architecture is the most crucial component for achieving OOD robustness in object detectors, with improvements in necks and heads having only marginal effects.
- Large-scale pre-training, especially using diverse datasets (e.g., ImageNet-21K, Objects365) and multi-modal supervision (vision-language models), significantly enhances OOD generalization.
- Detection Transformers (DETRs) exhibit surprisingly worse OOD generalization ability compared to traditional non-end-to-end detectors under natural distribution shifts, often showing negative Effective Robustness.
- Data augmentations like MixUp play a significant role in enhancing OOD robustness for object detectors.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.