Reproduce “Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zan Wang, Yixin Chen, Baoxiong Jia, Puhao Li, Jinlu Zhang, Jingze Zhang, Tengyu Liu, Yixin Zhu and 2 more

Researchers at BIGAI and collaborating institutions developed a two-stage diffusion framework that uses an intermediate scene affordance map to generate language-guided human motions accurately grounded in 3D environments. This approach significantly improves motion quality, physical plausibility, and demonstrates strong generalization to novel scenes and instructions, outperforming prior single-stage methods.

Key insights

  • Utilizing scene affordance as an intermediate representation effectively decouples complex multimodal generation, simplifying the learning problem and enhancing scene grounding.
  • Reframing affordance as a generalized distance field enables precise interaction grounding and robust generalization to novel scene geometries.
  • A two-stage diffusion framework significantly improves performance and generalization in language-guided human-scene interaction synthesis, especially when dealing with data scarcity.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.