Reproduce “Beyond ImageNet: Understanding Cross-Dataset Robustness of Lightweight Vision Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Weidong Zhang, Pak Lun Kevin Ding, Huan Liu
Arizona State University researchers introduced the Cross-Dataset Score (xScore) and a unified evaluation framework to quantify the real-world generalization of lightweight vision models across diverse datasets, finding that ImageNet performance is an incomplete predictor of cross-domain robustness and identifying architectural elements like hierarchical scaling and isotropic convolutions as crucial for consistent performance.
Key insights
- Cross-domain generalization capabilities vary significantly among lightweight architectures, even under identical training conditions and comparable parameter budgets.
- ImageNet accuracy (or a proxy like Imagenette-160) does not reliably predict performance on fine-grained or medical datasets, nor does it fully capture cross-dataset robustness.
- Architectural elements such as hierarchical scaling with Squeeze-and-Excite attention (EfficientNet) and isotropic convolutions with constant spatial resolution (ConvMixer) are critical drivers of robust generalization under resource constraints.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.