Reproduce “Real2Sim in HOI: Toward Physically Plausible HOI Reconstruction from Monocular Videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yubo Zhao, Yujin Chai, Yunao Dong, Chengfeng Zhao, Zijiao Zeng, Yuan Liu, Chi-Keung Tang
Researchers from The Hong Kong University of Science and Technology and Tencent IEG developed HA-HOI, a framework that reconstructs physically plausible 4D Human-Object Interaction (HOI) animations from monocular videos, enabling direct use in physics-based simulations. The method achieved a human Chamfer Distance of 7.11 cm and a human-object penetration score of 0.013 cm on the BEHAVE dataset, demonstrating superior physical consistency.
Key insights
- A human-centric formulation, where human motion serves as a stable world-grounded anchor, significantly enhances the coherence and robustness of HOI reconstruction from monocular videos.
- Explicitly incorporating VLM-guided semantic contact proposals and progressive optimization for contact and penetration dramatically reduces physically implausible interactions, bridging the 'interaction gap'.
- Integrating a physics-based projection stage with a residual controller allows kinematic reconstructions to be corrected and stabilized within a simulation environment, yielding truly 'simulation-ready' trajectories.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.