Reproduce “From Interpretability Methods to Interpretable Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Julien Colin, Nuria Oliver, Thomas Serre
This paper advocates for a fundamental shift in explainable AI (XAI) research, proposing a redirection of focus from developing and comparing explanation methods to systematically evaluating and comparing the interpretability of AI models themselves. It argues that the current XAI toolbox is mature enough and the AI landscape is ripe for this transition, which would advance human understanding of AI and help meet regulatory demands.
Key insights
- The current XAI toolbox, including attribution, feature visualization, concept-based, and circuit-based methods, is sufficiently mature to serve as reliable instruments for studying AI models.
- The field is at an opportune moment for this shift due to architectural convergence in computer vision models and the emergence of tools for fine-grained model comparison.
- Analogies to systems neuroscience suggest that significant scientific progress can be made by studying the object of interest (AI models) with available, albeit imperfect, tools, rather than waiting for "perfect" explanation methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.