Reproduce “Focal and Global Knowledge Distillation for Detectors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhendong Yang, Zhe Li, Xiaohu Jiang, Yuan Gong, Zehuan Yuan, Danpei Zhao, Chun Yuan
The Focal and Global Distillation (FGD) method is introduced to enhance knowledge transfer from large teacher models to smaller student models in object detection. It achieves significant performance gains, such as a 3.6 mAP improvement for FCOS-ResNet50 and 3.3 mAP for RetinaNet-ResNet50, by leveraging distinct focal and global distillation components.
Key insights
- Teacher and student feature maps exhibit uneven differences across foreground/background regions and channels, necessitating a differentiated, weighted approach to distillation.
- Distilling knowledge from both foreground and background areas is crucial, with background distillation primarily reducing false positives and foreground distillation improving target detection.
- The combination of fine-grained spatial and channel attention with global relational context is essential for comprehensive and effective knowledge transfer in object detection.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.