Reproduce “Text-driven Online Action Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Manuel Benavent-Lledo, David Mulero-Pérez, David Ortiz-Perez, Jose Garcia-Rodriguez
TOAD, a Text-driven Online Action Detection architecture from the University of Alicante, leverages frozen CLIP visual and textual embeddings to achieve efficient, semantically rich action detection. The method establishes new state-of-the-art performance with 82.46% mAP on the THUMOS14 dataset while also setting new baselines for zero-shot and few-shot online action detection.
Key insights
- Leveraging frozen pre-trained CLIP textual embeddings enables efficient vision-language model integration into online action detection, avoiding the high computational costs of contrastive learning.
- The integration of a novel future anticipation mechanism significantly enhances the accuracy of ongoing action detection by providing valuable contextual cues.
- The proposed architecture facilitates robust zero-shot and few-shot online action detection, greatly reducing the dependency on large, densely annotated datasets for new action categories.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.