Reproduce “Human Pose as Compositional Tokens” on OpenResearch

About this paper

Zigang Geng, Chunyu Wang, Yixuan Wei, Ze Liu, Houqiang Li, Han Hu

Human Pose as Compositional Tokens (PCT) introduces a discrete, structured representation for human poses, encoding interdependent joint sub-structures rather than independent joint positions. This approach significantly improves robustness and accuracy in occluded scenarios while achieving competitive performance on standard benchmarks and streamlining the pose estimation pipeline by eliminating expensive post-processing.

Key insights

• Representing poses as discrete, compositional tokens encoding interdependent sub-structures fundamentally improves anatomical plausibility and robustness, particularly under severe occlusion.
• Jointly learning a shared codebook of sub-structures (tokens) and forcing reconstruction from masked inputs helps the model infer missing joint information and reduces reliance on direct visual cues.
• Transforming pose estimation into a classification problem over a discrete latent space simplifies the overall pipeline by eliminating the need for complex, hand-engineered post-processing steps common in heatmap-based methods.

Summary by alphaXiv.