Reproduce “ProtoPFormer: Concentrating on Prototypical Parts in Vision Transformers for Interpretable Image Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mengqi Xue, Qihan Huang, Haofei Zhang, Jingwen Hu, Jie Song, Mingli Song, Canghong Jin
ProtoPFormer introduces a Vision Transformer-based architecture that integrates global and local prototypes for interpretable image recognition, effectively addressing the "prototype distraction" issue to yield superior classification accuracy and transparent visual explanations concentrated on discriminative object parts.
Key insights
- ViT's intrinsic architectural components, such as the class token and attention mechanisms, can be effectively leveraged to guide prototype learning and foreground identification.
- Explicitly enforcing prototype concentration on discriminative parts and foreground elements, through tailored masking and loss functions, is crucial for overcoming the "prototype distraction" problem in ViTs.
- Combining global contextual understanding with local fine-grained part recognition via a dual-prototype approach enables mutual correction and more robust, interpretable classification decisions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.