Reproduce “MPL: Lifting 3D Human Pose from Multi-view 2D Poses” on OpenResearch

About this paper

Seyed Abolfazl Ghasemzadeh, Alexandre Alahi, Christophe De Vleeschouwer

This work introduces MPL, a multi-view 3D pose lifter designed for robust estimation in unconstrained environments, which decouples 2D pose estimation from 3D lifting. It achieves a 56% reduction in Mean Per Joint Position Error over triangulation for 2 views on Human3.6M by training on a novel synthetically generated dataset that simulates real-world 2D pose inaccuracies.

Key insights

• Decoupling 2D pose estimation from 3D lifting allows leveraging robust, pre-trained 2D estimators and training the 3D lifter solely on synthetic data, circumventing the need for costly real-world 3D annotations.
• Generating synthetic 2D-3D training data by applying the target 2D pose estimator to rendered 3D meshes injects realistic noise and ensures consistent keypoint definitions, significantly enhancing the 3D lifter's robustness to real-world 2D inaccuracies and varying acquisition setups.
• A transformer-based architecture effectively fuses information from multiple views and across anatomical keypoints, leading to a more accurate and robust 3D pose reconstruction compared to simpler network designs or direct triangulation.

Summary by alphaXiv.