Reproduce “Few-Shot Video Object Detection” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Qi Fan, Chi-Keung Tang, Yu-Wing Tai

Researchers introduced Few-Shot Video Object Detection (FSVOD), a new task focusing on detecting novel object classes in videos given only a few example images. Their method, which combines a Tube Proposal Network (TPN) and a Temporal Matching Network (TMN+), achieved 25.1 AP on the new FSVOD-500 test set, outperforming image-based few-shot object detection by 4.0 AP.

Key insights

  • Leveraging temporal coherence with a Tube Proposal Network (TPN) is crucial for generating discriminative object proposals in videos, reducing false positives and aggregating robust features across frames.
  • Aligning the feature aggregation process between training and inference phases in the matching network (via a Temporal Alignment Branch) significantly improves generalization to novel classes in videos.
  • Enhancing the matching network with Label-Smoothing Regularization and a Support Classification Module improves the discriminative power and representativeness of features, leading to better performance on novel categories.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.