Reproduce “Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Video” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zongmian Li, Jiri Sedlar, Justin Carpentier, Ivan Laptev, Nicolas Mansard, Josef Sivic
A method was developed to reconstruct 3D motion, contact positions, and interaction forces of people manipulating objects from monocular RGB video by integrating 2D visual cues with physics-based trajectory optimization. This approach achieved a 3D human motion accuracy (MPJPE) of 122.11 mm on the Parkour dataset and enhanced 2D object endpoint localization by up to 47% on a new Handtool dataset.
Key insights
- Jointly estimating 3D trajectories of the person and object, contact positions, and interaction forces, linked by physical laws, effectively resolves ambiguities inherent in vision-only approaches.
- Automatically recognizing the presence, position, and timing of contacts directly from input video provides crucial constraints, simplifying the complex, non-smooth nature of contact-aware optimal control.
- The integration of vision-based 2D cues with a physics-based optimal control framework enables the reconstruction of physically plausible and dynamically consistent human-object interactions from monocular video.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.