Reproduce “DAVE: Distribution-aware Attribution via ViT Gradient Decomposition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Adam Wróbel, Siddhartha Gairola, Jacek Tabor, Bernt Schiele, Bartosz Zieliński, Dawid Rymarczyk
DAVE introduces a method for generating stable, high-resolution attribution maps for Vision Transformers by decomposing the input gradient to isolate and suppress architecture-induced noise and local instabilities. This approach improves localization metrics by up to +11.92 percentage points and enhances explanation faithfulness across various ViT architectures and even for inherently interpretable B-cos models.
Key insights
- Decomposing the input gradient in ViTs allows for the isolation and removal of unstable, high-frequency "operator variation" components, leading to cleaner attribution signals.
- Enforcing local equivariance on the effective transformation, through averaging over small spatial perturbations, is key to filtering out architecture-specific artifacts (like patch grids) while preserving semantic content.
- A combination of gradient decomposition, local equivariance, and low-pass filtering results in attribution maps that are both high-resolution and remarkably stable, addressing a long-standing trade-off in ViT XAI.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.