Reproduce “Sequence Level Semantics Aggregation for Video Object Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haiping Wu, Yuntao Chen, Naiyan Wang, Zhaoxiang Zhang
The Sequence Level Semantics Aggregation (SELSA) module integrates into object detectors, aggregating features from semantically similar proposals across entire video sequences to improve video object detection. This method enhances robustness against object appearance degradation, achieving 84.30 mAP on ImageNet VID with ResNeXt-101 while reducing reliance on complex post-processing steps.
Key insights
- Treating video as a "bag of unordered frames" and leveraging full-sequence semantic similarities provides more robust feature enhancement than relying solely on temporal proximity or motion estimation.
- Aggregating features based on semantic similarity at the sequence level effectively reduces intra-class feature variance for objects, improving detection performance under appearance degradation.
- An end-to-end differentiable module for sequence-level semantic aggregation can intrinsically capture video-level context, largely negating the need for external, often sub-optimal, post-processing techniques.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.