Reproduce “Rethinking pose estimation in crowds: overcoming the detection information-bottleneck and ambiguity” on OpenResearch

About this paper

Mu Zhou, Lucas Stoffl, Mackenzie Weygandt Mathis, Alexander Mathis

Researchers at EPFL developed the Bottom-Up Conditioned Top-Down (BUCTD) framework for multi-instance pose estimation, specifically in crowded and occluded environments, achieving new state-of-the-art performance on benchmarks like CrowdPose (78.5 AP) and OCHuman (48.5 AP) by overcoming information bottlenecks in traditional methods.

Key insights

• Leveraging bottom-up pose estimators as the initial 'detection' stage provides pose-aware conditional input, effectively resolving the information bottleneck inherent in generic object detection.
• The conditional input acts as a 'pointing' mechanism, enabling top-down models to disambiguate and accurately refine the pose of a specific individual within potentially crowded bounding boxes.
• Training CTD models with diverse, noisy pose conditions (empirically sampled from bottom-up predictions or generatively augmented) enhances robustness and generalization to various initial detection errors.

Summary by alphaXiv.