Reproduce “Are Pose Estimators Ready for the Open World? STAGE: A GenAI Toolkit for Auditing 3D Human Pose Estimators” on OpenResearch

About this paper

Nikita Kister, István Sárándi, Jiayi Wang, Anna Khoreva, Gerard Pons-Moll

Researchers from Bosch, University of Tübingen, Zalando SE, and Max Planck Institute for Informatics developed STAGE, a GenAI toolkit for auditing 3D human pose estimators. This toolkit generates controlled image pairs that isolate specific attributes, enabling quantification of model robustness and revealing sensitivities previously difficult to assess with existing benchmarks.

Key insights

• Leveraging isolated human-body depth maps (SMPL-based), dense semantic encoding, and a mixed training strategy (3D and 2D pose datasets) enables generative models to produce photorealistic images with precise 3D pose control and diverse attribute variation.
• Generating image pairs by varying only a text prompt attribute while sharing the generative model's initial noise effectively isolates the impact of a single attribute on pose estimation, reducing confounding factors.
• The Percentage of Degraded Poses (PDP) metric provides a quantifiable measure of an estimator's robustness, specifically highlighting performance degradation under specific attribute changes rather than overall average error.

Summary by alphaXiv.