Reproduce “SeaFormer++: Squeeze-enhanced Axial Transformer for Mobile Visual Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qiang Wan, Zilong Huang, Jiachen Lu, Gang Yu, Li Zhang
A new mobile-friendly Vision Transformer, SeaFormer++, utilizes Squeeze-enhanced Axial Attention and a feature upsampling-based multi-resolution distillation to achieve superior accuracy and efficiency for on-device visual recognition. It delivers an 8.3% mIoU gain over MobileNetV3 on ADE20K with 16% lower latency, and reduces inference latency by up to 50% with minimal performance loss via distillation.
Key insights
- The novel Squeeze-enhanced Axial Attention (SEA attention) block efficiently extracts global semantics while preserving local details, achieving linear computational complexity with feature map size.
- Adaptive squeeze and expand methods, utilizing learnable masks, significantly improve performance (1.0% mIoU/Top1 accuracy gain) over traditional pooling/broadcast in efficient attention mechanisms.
- Multi-resolution distillation, guided by feature upsampling, allows for substantial latency reduction (e.g., 50%) for mobile inference without significant performance degradation by transferring knowledge from high-resolution teachers to low-resolution student models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.