Reproduce “Problems with Shapley-value-based explanations as feature importance measures” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
I. Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, Sorelle Friedler
This paper critically examines the suitability of Shapley-value-based explanations as reliable feature importance measures in machine learning, demonstrating that their mathematical elegance does not consistently translate to effective or meaningful insights for complex models. It details how these methods face fundamental challenges related to handling feature dependencies, evaluating out-of-distribution data, and aligning with human needs for actionable and contrastive explanations.
Key insights
- Shapley values for feature importance face a dilemma: conditional methods suffer from computational complexity and sensitivity to feature redundancy, while interventional methods rely on out-of-distribution samples leading to arbitrary explanations.
- The additivity axiom of Shapley values can render them uninformative for non-additive models, as they may attribute equal influence to all features regardless of individual impact.
- Shapley-based explanations often do not align with human preferences for contrastive explanations and offer limited utility for providing actionable recourse or aiding normative evaluation, frequently leading to misinterpretations and over-trust among practitioners.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.