Reproduce “Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhengdi Yu, Stefanos Zafeiriou, Tolga Birdal
Dyn-HaMR recovers global 4D motion of two hands from monocular video captured by a moving camera. It estimates camera motion, fits hand observations, and regularizes sequences with a learned hand-motion prior and interaction constraints. On H2O’s egocentric view, it reports 45.6 mm G-MPJPE and 34.2 mm GA-MPJPE, versus HaMeR’s 96.9 and 75.7 mm, under the evaluation protocol.
Key insights
- Explicitly disentangling hand and camera motion through SLAM-estimated camera poses and a learnable world scale factor is crucial for accurate 4D global hand trajectory recovery from dynamic monocular video.
- Integrating a data-driven Hand Motion Prior (HMP) with biomechanical and interpenetration constraints enables robust and plausible reconstruction of complex interacting hand motions, even under significant occlusions.
- A multi-stage optimization framework, beginning with hierarchical initialization and progressing to global motion and interaction-specific refinements, effectively addresses the combined challenges of dynamic cameras and bimanual interactions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.