Reproduce “Aggregation Hides Out-of-Distribution Generalization Failures from Spurious Correlations” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Olawale Salaudeen, Haoran Zhang, Kumail Alhamoud, Sara Beery, Marzyeh Ghassemi

An investigation into out-of-distribution (OOD) generalization benchmarks demonstrates that standard aggregate metrics frequently obscure subsets where models relying on spurious correlations exhibit worse OOD performance despite higher in-distribution accuracy. Researchers at MIT developed OODSelect to systematically uncover these "accuracy-on-the-inverse-line" (AoTIL) phenomena across various datasets.

Key insights

  • The commonly observed "accuracy-on-the-line" (AoTL) phenomenon in aggregate OOD benchmarks is often an artifact of heterogeneous example aggregation, masking critical generalization failures.
  • AoTIL subsets represent systematic generalization failures driven by models' reliance on specific spurious correlations, rather than simply reflecting generally difficult or noisy examples.
  • Current aggregate OOD evaluation metrics provide an incomplete and potentially misleading picture of model robustness, underscoring the necessity for more refined evaluation methodologies.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.