Reproduce “Feature Fusion Vision Transformer for Fine-Grained Visual Categorization” on OpenResearch

About this paper

Jun Wang, Xiaohan Yu, Yongsheng Gao

Researchers at the University of Warwick and Griffith University developed the Feature Fusion Vision Transformer (FFVT) to enhance fine-grained visual categorization by fusing discriminative tokens from multiple transformer layers. This method achieved 91.5% accuracy on Stanford Dogs and demonstrated superior performance on challenging small-scale, ultra-fine-grained datasets like CottonCultivar80 with 57.92% accuracy.

Key insights

• Fusing multi-level local and low-level features significantly enhances the discriminative power of Vision Transformers for fine-grained tasks, as deep layers often prioritize global context.
• Bidirectional attention-based token selection (MAWS) effectively identifies crucial discriminative features without additional parameters, outperforming simpler attention-based selection methods.
• Pure transformer architectures, when adapted with feature fusion, can achieve state-of-the-art performance in fine-grained visual categorization, even on challenging small-scale and ultra-fine-grained datasets.

Summary by alphaXiv.