Reproduce “Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainability” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Joakim Edin, Andreas Geert Motzfeldt, Casper L. Christensen, Tuukka Ruotsalo, Lars Maaløe, Maria Maistro

Research by Edin et al. reveals a critical flaw in the widely used AOPC faithfulness metric, demonstrating that its scores are inherently incomparable across different deep learning models due to varying theoretical limits. The team introduces Normalized AOPC (NAOPC), a min-max normalization technique, which re-establishes valid cross-model comparisons and significantly alters model faithfulness rankings.

Key insights

  • The study empirically proves that the optimal (lower and upper) AOPC scores are highly variable across different deep neural network models and inputs, making unnormalized scores incomparable.
  • Normalization using NAOPC radically changes the relative faithfulness rankings of models, challenging conclusions from prior research that relied on unnormalized AOPC for cross-model comparisons.
  • NAOPC largely preserves the relative faithfulness rankings of different feature attribution methods when evaluated within a single model, suggesting its primary benefit is for cross-model interpretability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.