Reproduce “SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yongle Huang, Haodong Chen, Zhenbang Xu, Zihan Jia, Haozhou Sun, Dian Shao
SeFAR, a framework for semi-supervised fine-grained action recognition, introduces a dual-level temporal modeling approach, a moderate temporal perturbation strategy, and an adaptive regulation mechanism to achieve state-of-the-art performance with limited labeled data and enhance the fine-grained understanding capabilities of Multimodal Large Language Models.
Key insights
- Capturing action information at both fine-grained (short segments) and contextual (longer segments) temporal scales is crucial for effectively distinguishing subtle variations in FAR.
- A tailored temporal perturbation strategy, that distorts local temporal order but maintains global context, proves more effective for video-based strong augmentation in SSL than generic methods.
- Dynamically adjusting the unsupervised loss based on the Teacher model's prediction confidence and uncertainty significantly improves training stability and performance in challenging semi-supervised fine-grained tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.