Reproduce “On the Faithfulness of Vision Transformer Explanations” on OpenResearch

About this paper

Junyi Wu, Weitai Kang, Hao Tang, Yuan Hong, Yan Yan

Researchers at the Illinois Institute of Technology, Carnegie Mellon University, and the University of Connecticut developed the Salience-guided Faithfulness Coefficient (SaCo), a novel metric for Vision Transformer explanations. SaCo quantitatively evaluates faithfulness by comparing the magnitude of pixel salience scores with their actual impact on model predictions, revealing that existing metrics often fail to distinguish meaningful explanations from random noise and showing how gradient information and multi-layer aggregation enhance explanation faithfulness.

Key insights

• SaCo measures a distinct aspect of explanation quality with very low correlation (0.18-0.22) to existing metrics, indicating that it captures unique information about faithfulness.
• SaCo consistently scores "Random Attribution" near zero across all experiments, demonstrating its ability to robustly distinguish meaningful explanations from noise, a critical failure point for traditional metrics.
• Both the incorporation of gradient information (e.g., ~106% boost in SaCo score for last-layer attention) and multi-layer aggregation significantly improve the faithfulness of attention-based ViT explanations.

Summary by alphaXiv.