Reproduce “ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimation” on OpenResearch

About this paper

Zehao Zhu, Jiashun Wang, Yuzhe Qin, Deqing Sun, Varun Jampani, Xiaolong Wang

Researchers from the University of Texas at Austin, Carnegie Mellon University, UC San Diego, and Google Research developed ContactArt, a framework that leverages a scalable synthetic dataset and learned geometric interaction priors to improve 3D hand and articulated object pose estimation. This approach achieves state-of-the-art performance on real-world datasets, reducing mean per-vertex hand pose error from 71.6mm to 49.9mm and demonstrating robust generalization by leveraging transferable articulation and contact priors from simulation.

Key insights

• The geometric consistency of hand-object contacts and object articulation transfers more effectively from simulation to reality than visual appearance, enabling robust sim2real generalization.
• Scalable and accurate 3D interaction data, including precise contact points, can be efficiently generated through visual teleoperation in a physical simulator, addressing the costly data annotation challenge.
• Advanced generative models (GAN discriminators for articulation and diffusion models for contact) can effectively learn and enforce critical 3D geometric priors, enhancing the realism and accuracy of pose estimations.

Summary by alphaXiv.