Reproduce “Vanilla Group Equivariant Vision Transformer: Simple and Effective” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiahong Fu, Qi Xie, Deyu Meng, Zongben Xu
A modular recipe turns standard ViT and Swin Transformer components into networks equivariant to discrete rotations and reflections. Across classification, detection, segmentation, image super-resolution, and video super-resolution, the resulting models match or improve baseline accuracy while sometimes using substantially fewer parameters, including an ImageNet ViT-B variant reaching 80.28% top-1 accuracy with 38M parameters versus 77.18% with 87M for the baseline.
Key insights
- Exact equivariance requires an end-to-end treatment of the Transformer pipeline; modifying self-attention alone is insufficient when other modules discard geometric correspondence.
- The framework supports groups such as C4 and D4, covering 90-degree rotations and mirror reflections, and can be incorporated into existing vision architectures with relatively direct module substitutions.
- Theoretical analysis links group-wise parameter sharing to lower model complexity and potentially better generalization, although the practical strength of this relationship depends on the assumptions behind the bound.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.