Reproduce “Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes” on OpenResearch

About this paper

Jiashun Wang, Huazhe Xu, Jingwei Xu, Sifei Liu, Xiaolong Wang

Researchers from UC San Diego, UC Berkeley, Shanghai Jiao Tong University, and NVIDIA developed a framework for synthesizing long-term 3D human motion that explicitly incorporates detailed human-scene interactions within 3D environments. This approach generates motions that are more physically plausible and natural, achieving average naturalness scores from human evaluators between 3.39 and 3.65 on a 1-5 scale, nearing pseudo-ground truth realism.

Key insights

• Decomposing long-term motion synthesis into hierarchical stages—static sub-goal pose generation followed by short-term motion interpolation—effectively addresses the complexity of generating extended human movements.
• Integrating a differentiable human body model (SMPL-X) with post-generation geometric optimization allows for precise enforcement of physical interaction constraints, such as collision avoidance and contact modeling, enhancing realism within 3D scenes.
• Explicitly conditioning the generative process on 3D scene structure and affordances, along with incorporating diverse body shapes via CVAE, leads to more natural, context-aware, and varied human motions.

Summary by alphaXiv.