Reproduce “A Deep Dive into Dataset Imbalance and Bias in Face Identification” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Valeriia Cherepanova, Steven Reich, Samuel Dooley, Hossein Souri, Micah Goldblum, Tom Goldstein

This research dissects how different types of dataset imbalance in both training and testing data affect bias in face identification systems. It demonstrates that these imbalances can interact in counter-intuitive ways, potentially masking actual model bias during evaluations, and reveals performance changes across various demographic imbalance scenarios.

Key insights

  • Training and testing data imbalances interact complexly; for instance, opposing effects of identity imbalance can cancel out, leading to an underestimation of true model bias during evaluations, analogous to Simpson's paradox.
  • Increasing the proportion of identities from a demographic group in the test gallery surprisingly harms the model's performance for that group, while increasing the number of images per identity benefits performance.
  • Even randomly initialized (untrained) neural networks exhibit demographic bias, suggesting that intrinsic image characteristics or architectural properties contribute to bias beyond just training data distribution.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.