Reproduce “BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ivan Grishchenko, Valentin Bazarevsky, Andrei Zanfir, Eduard Gabriel Bazavan, Mihai Zanfir, Richard Yee, Karthik Raveendran, Matsvei Zhdanovich and 2 more
A lightweight neural network pipeline developed by Google provides real-time 3D human body and hand landmarks along with full body pose and shape parameters from a single RGB image. It achieves high accuracy (e.g., 78mm MPJPE-PA) and runs efficiently on mobile phones and web browsers, democratizing advanced motion capture capabilities.
Key insights
- Leveraging a statistical 3D human model (GHUM) combined with depth ordering supervision allows for scalable generation of high-quality 3D ground truth from existing 2D datasets.
- A clever re-cropping strategy for hand regions enables precise, high-resolution 3D hand landmark estimation within an overall low-resolution body pose pipeline without significantly increasing computational load.
- An efficient MLP-Mixer architecture can effectively map predicted 3D landmarks to semantically rich 3D human model parameters (GHUM pose and shape) for real-time applications.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.