Reproduce “Exploring Adaptive Masked Reconstruction for Self-Supervised Skeleton-Based Action Recognition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shengkai Sun, Zhiyong Cheng, Zefan Zhang, Jianfeng Dong, Zhihui Li, Meng Wang
An Adaptive Masked Reconstruction (AMR) framework improves self-supervised skeleton-based action recognition by enabling efficient large-patch reconstruction and adaptively focusing on critical motion patterns. The method demonstrates an 8.1x training speedup and achieves higher accuracy on NTU-60, NTU-120, and PKU-II datasets compared to prior state-of-the-art approaches.
Key insights
- Replacing the self-attention among mask tokens in the decoder with a purely cross-attention design can efficiently reconstruct larger spatiotemporal patches without performance degradation.
- Not all masked regions contribute equally to action semantics; adaptively focusing reconstruction efforts on areas with higher motion energy leads to more discriminative feature learning.
- Quantifying motion energy across multiple temporal scales provides a more robust and continuous measure of joint importance for guiding reconstruction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.