Reproduce “End-to-end Temporal Action Detection with Transformer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaolong Liu, Qimeng Wang, Yao Hu, Xu Tang, Shiwei Zhang, Song Bai, Xiang Bai
TadTR presents an end-to-end Transformer-based framework for temporal action detection, directly predicting action segments without hand-crafted components. This approach achieves state-of-the-art accuracy on THUMOS14, HACS Segments, and ActivityNet-1.3 while demonstrating significantly faster inference speeds and lower computational costs compared to prior methods.
Key insights
- Adapting the Transformer architecture for temporal action detection as a direct set prediction problem simplifies the pipeline and enables true end-to-end learning.
- Temporal Deformable Attention (TDA) is essential for efficient and effective context modeling in video sequences, addressing high temporal redundancy and subtle changes.
- Combining incremental segment refinement with an actionness regression head yields more accurate temporal boundaries and reliable detection confidence scores.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.