Reproduce “World-Coordinate Human Motion Retargeting via SAM 3D Body” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhangzheng Tum, Kailun Su, Shaolong Zhu, Yukun Zheng
An engineering-oriented framework reconstructs physically plausible, world-coordinate human motion from monocular videos by integrating SAM 3D Body and Momentum Human Rig with temporal smoothing and contact-aware global root optimization. This approach produces stable world trajectories and reliably retargets motion to humanoid robots like the Unitree G1, addressing challenges in vision-to-robot translation.
Key insights
- Robust world-coordinate human motion for robotics can be achieved by effectively integrating powerful, off-the-shelf foundation models (like SAM 3D Body and MHR) without extensive re-training.
- Lightweight physical constraints (e.g., foot contact modeling) and structured kinematic representations (MHR's invariant bone lengths) are more suitable for rigid-link robot retargeting than complex temporal networks or models like SMPL.
- Temporal consistency can be efficiently enforced through latent-space smoothing and trajectory-level identity constraints, bridging the gap between noisy monocular observations and stable robot-ready motion.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.