Reproduce “Impossibility Theorems for Feature Attribution” on OpenResearch

About this paper

Blair Bilodeau, Natasha Jaques, Pang Wei Koh, Been Kim

Researchers from the University of Toronto, University of Washington, and Google DeepMind demonstrated theoretical impossibility theorems for a class of complete and linear feature attribution methods, including SHAP and Integrated Gradients. This work shows that these widely used methods cannot reliably infer local counterfactual model behavior, performing no better than random guessing for tasks such as algorithmic recourse and spurious feature identification.

Key insights

• Satisfying desirable axiomatic properties like completeness and linearity does not guarantee practical reliability for inferring how local feature changes affect model outputs.
• Complete and linear feature attribution methods, such as SHAP and Integrated Gradients, can perform no better than random guessing for critical interpretability tasks.
• The inherent difficulty lies with the current attribution methods, not the interpretability problem itself, as direct model querying (a 'brute-force' approach) can reliably infer counterfactual behavior.

Summary by alphaXiv.