Reproduce “LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Akshita Gupta, Gaurav Mittal, Ahmed Magooda, Ye Yu, Graham W. Taylor, Mei Chen
LoSA introduces a memory- and parameter-efficient adapter that enables end-to-end backbone adaptation of billion-parameter video foundation models for temporal action localization in untrimmed videos. This approach achieves state-of-the-art performance on THUMOS-14 and ActivityNet-v1.3 datasets, making it feasible to leverage large models for complex video understanding tasks.
Key insights
- Capturing both short-range (fine-grained) and long-range (global) temporal dependencies is essential for accurate temporal action localization in untrimmed videos.
- An adaptive, gated fusion mechanism for combining multi-range temporal features from various backbone layers yields higher performance than simpler aggregation methods.
- Memory-and-parameter-efficient parallel adapter designs can overcome GPU memory limitations, enabling end-to-end adaptation for billion-parameter video foundation models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.