Reproduce “Attention-guided Feature Distillation for Semantic Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Amir M. Mansourian, Arya Jalali, Rozhan Ahmadi, Shohreh Kasaei
Researchers from Sharif University of Technology developed AttnFD, a method for knowledge distillation in semantic segmentation that refines intermediate feature maps using a Convolutional Block Attention Module (CBAM) before applying a simple Mean Squared Error (MSE) loss. This approach achieved state-of-the-art performance across Pascal VOC 2012, Cityscapes, COCO, and CamVid datasets, boosting mIoU for DeepLabV3-ResNet18 student on Pascal VOC 2012 by 5.59% over baseline and outperforming prior methods by 1.67%.
Key insights
- Refining intermediate feature maps with attention mechanisms prior to distillation makes knowledge transfer more effective, allowing for the use of a simpler distillation loss.
- The Convolutional Block Attention Module (CBAM) is particularly effective for feature refinement, offering a strong balance between performance gains and computational cost by integrating both channel and spatial attention.
- Distilling knowledge from semantically richer layers, especially decoder features, yields the most pronounced improvements in student network performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.