Reproduce “BBoxMaskPose v2: Expanding Mutual Conditioning to 3D” on OpenResearch

About this paper

Miroslav Purkrabek, Constantin Kolomiiets, Jiri Matas

BBoxMaskPose v2 (BMPv2) iteratively refines detections, masks, and poses by conditioning each representation on the others. Its PMPose model uses masks for pose estimation, while SAM-pose2seg uses pose prompts to refine segmentation; BMPv2+ adds a pose–mask refinement pass. On OCHuman, BMPv2+ reaches 55.8 pose AP, versus 49.2 for BMPv1, at greater cost; pose AP measures 2D similarity, not 3D accuracy.

Key insights

• An iterative framework that mutually conditions detection, pose estimation, and segmentation, utilizing improved components, effectively addresses the challenges of multi-person interaction in crowded scenes.
• Probabilistic modeling in pose estimation (PMPose) provides richer outputs like keypoint presence, visibility, and expected OKS, crucial for accurate limb assignment, border stability, and confidence calibration.
• Specialized fine-tuning of foundational models like SAM for domain-specific tasks, coupled with pose-guided prompting, significantly enhances their utility and precision for human instance segmentation.
• High-quality 2D outputs derived from robust 2D pipelines directly and substantially improve downstream 3D human pose estimation, even in challenging crowded environments.

Summary by alphaXiv.