Reproduce “DECO: Dense Estimation of 3D Human-Scene Contact In The Wild” on OpenResearch

About this paper

Shashank Tripathi, Agniv Chatterjee, Jean-Claude Passy, Hongwei Yi, Dimitrios Tzionas, Michael J. Black

Researchers from Max Planck Institute for Intelligent Systems and the University of Amsterdam introduce DAMON, a large-scale dataset of dense, vertex-level 3D human-scene contact annotations from in-the-wild images, and DECO, a model for estimating 3D contact using body-part and scene context. DECO achieved higher F1 scores against geometric methods and improved 3D human pose and shape estimation by reducing Vertex-to-Vertex error from 183.3 mm to 171.6 mm.

Key insights

• Dense, vertex-level 3D contact is fundamentally a 3D phenomenon, requiring dedicated 3D annotations beyond current 2D or coarse 3D approaches to accurately represent human-scene interactions.
• Robustly inferring occluded 3D contact in unconstrained environments necessitates explicitly reasoning about both body-part-specific cues and the broader scene context, effectively integrated through cross-attention mechanisms.
• Direct inference of 3D human-scene contact from images can be more accurate and efficient than deriving it geometrically from state-of-the-art 3D human and object reconstructions.

Summary by alphaXiv.