Reproduce “Focus! Rating XAI Methods and Finding Biases” on OpenResearch

About this paper

Anna Arias-Duart, Ferran Parés, Dario Garcia-Gasulla, Victor Gimenez-Abalos

A new quantitative metric called Focus is introduced to objectively evaluate feature attribution methods in image classification using specially constructed mosaic images. This metric not only assesses the coherency of XAI methods to model predictions but also serves as a tool for identifying and characterizing underlying model biases.

Key insights

• The use of in-distribution mosaic images allows for objective, quantitative assessment of XAI method faithfulness, overcoming limitations of prior evaluation techniques that introduced out-of-distribution artifacts.
• Errors in XAI attributions on these mosaics directly reveal and characterize specific model biases, such as "shared bias" (visual features incorrectly linked to a different class) and "missing bias" (over-reliance on a single feature).
• The fidelity of different XAI methods varies significantly across architectures and datasets, with some methods providing much more coherent explanations than others.

Summary by alphaXiv.