Reproduce “Prompted Contrast with Masked Motion Modeling: Towards Versatile 3D Action Representation Learning” on OpenResearch

About this paper

Jiahang Zhang, Lilang Lin, Jiaying Liu

PCM³ combines contrastive learning and topology-aware masked motion modeling to learn 3D skeleton representations that capture both action-level semantics and fine-grained joint dynamics. Across recognition, retrieval, occlusion robustness, detection, and motion prediction benchmarks, the method improves transfer performance over separate or naively combined pretraining objectives, reaching up to 93.1% recognition accuracy and 101.7 mm MPJPE.

Key insights

• Masked modeling contributes local spatial-temporal structure, while contrastive learning supplies discriminative sequence-level semantics; their explicit bidirectional interaction is more effective than simple multi-task learning.
• Masking body regions rather than isolated joints creates stronger reconstruction challenges and yields better transfer, with 60% topology masking outperforming random masking in ablations.
• The learned representation supports both semantic tasks and structured prediction, suggesting that a single skeleton encoder can transfer across recognition, retrieval, detection, and motion forecasting.

Summary by alphaXiv.