Reproduce “Fidelity of Interpretability Methods and Perturbation Artifacts in Neural Networks” on OpenResearch

About this paper

Lennart Brocki, Neo Christopher Chung

A framework from the University of Warsaw quantifies the impact of perturbation artifacts on the fidelity evaluation of neural network interpretability methods, revealing that these artifacts contribute systematically small errors relative to overall fidelity and enabling a more robust ranking of saliency techniques such as Smoothgrad and Integrated Gradient.

Key insights

• Perturbation artifacts, a known confound in evaluating interpretability methods, can be systematically quantified and separated from the actual information removal.
• The estimated systematic errors due to artifacts are consistently small relative to the measured fidelity, allowing for a more reliable and robust ranking of interpretability methods.
• Smoothgrad and Squared SmoothGrad consistently demonstrate higher fidelity in identifying important features compared to Integrated Gradient and Vanilla Gradient, aligning with qualitative assessments of saliency maps.

Summary by alphaXiv.