Reproduce “Iterative Orthogonal Feature Projection for Diagnosing Bias in Black-Box Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Julius Adebayo, Lalana Kagal

Researchers at MIT CSAIL developed an iterative orthogonal feature projection method to quantify how much black-box predictive models rely on specific input attributes for their decisions. Applying this approach to a bank's credit limit algorithm demonstrated low predictive dependence on gender and higher dependence on factors like age and income.

Key insights

  • Orthogonal projection offers a mathematically grounded approach to isolate the impact of individual features on model output, allowing for bias diagnosis without internal model access.
  • Augmenting input data with non-linear transformations (polynomial, logarithmic, exponential) prior to projection helps capture non-linear dependencies common in black-box models.
  • Quantifying the change in model performance after feature projection provides an objective and interpretable metric for assessing a model's fairness and identifying potential discriminatory reliance on attributes.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.