Reproduce “CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhihao Li, Jianzhuang Liu, Zhensong Zhang, Songcen Xu, Youliang Yan
CLIFF reconstructs 3D human body meshes from person crops while retaining each crop’s location and size in the full-image bounding box. It also projects predicted joints onto the full image for reprojection supervision. On 3DPW, CLIFF with ResNet-50 reports MPJPE of 72.0 mm and PVE of 85.3 mm, versus HMR-EFT at 85.1 mm and 98.7 mm; these results are from one benchmark.
Key insights
- Global location and size information from the full image frame are crucial for accurate global rotation and articulated pose estimation in 3D human mesh recovery, which is often lost in top-down cropping approaches.
- Supervising 2D reprojection loss in the original full image plane provides more accurate and globally-aware guidance for 3D pose predictions than doing so within cropped images.
- A robust pseudo-ground-truth annotator built on a globally-aware model can effectively bridge the data gap for 3D supervision, generating high-quality annotations that significantly improve downstream model performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.