Reproduce “SkateFormer: Skeletal-Temporal Transformer for Human Action Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jeonghyeok Do, Munchurl Kim
SkateFormer, a new transformer-based architecture from KAIST, efficiently and effectively captures diverse skeletal-temporal relations for human action recognition, achieving state-of-the-art performance on benchmarks like NTU RGB+D while demonstrating a significant reduction in computational complexity.
Key insights
- Human actions involve varied spatio-temporal dynamics that necessitate adaptive modeling of both local and global interactions between neighboring and distant joints across different temporal scales.
- Structured, partition-specific attention can achieve significant computational efficiency while effectively capturing complex, data-adaptive skeletal-temporal relationships.
- Combining learnable skeletal positional embeddings with fixed temporal embeddings via an outer product, along with novel data augmentations, enhances robustness and generalization in action recognition.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.