Reproduce “Benchmarking Low-Shot Robustness to Natural Distribution Shifts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aaditya Singh, Kartik Sarangmath, Prithvijit Chattopadhyay, Judy Hoffman
The authors evaluate pretrained image classifiers and robustness interventions under low-shot training across three benchmarks and natural shifts. They compare rankings and intervention effects across datasets and training regimes. Rankings vary by dataset, and full-shot gains often fail to persist. On Camelyon, DINO ViT-B/16 achieves 86.93% in-domain and 84.33% out-of-domain accuracy; these results do not guarantee performance on other shifts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.