Reproduce “HAP: Structure-Aware Masked Image Modeling for Human-Centric Perception” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junkun Yuan, Xinyu Zhang, Hao Zhou, Jian Wang, Zhongwei Qiu, Zhiyin Shao, Shaofeng Zhang, Sifan Long and 7 more
HAP, a self-supervised pre-training framework, introduces Masked Image Modeling (MIM) to human-centric perception by integrating human structure priors and a structure-invariant alignment loss. This approach achieves new state-of-the-art results on 11 out of 12 benchmarks across five diverse human-centric tasks, including person ReID and 2D/3D human pose estimation.
Key insights
- Adapting MIM with human-centric domain knowledge, such as optimized scaling and masking ratios and explicit human part priors, is crucial for its effectiveness in human-centric perception.
- Learning invariance to structural corruption through feature alignment of different masked views significantly enhances the discriminative power of the learned representations.
- A plain Vision Transformer encoder, when pre-trained with HAP on a large-scale human dataset, can serve as a robust and versatile foundational model for various human-centric tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.