Reproduce “Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuan Wang, Zhao Wang, Junhao Gong, Di Huang, Tong He, Wanli Ouyang, Jile Jiao, Xuetao Feng and 3 more
Researchers introduce "Holistic-Motion2D," a large-scale 2D whole-body human motion dataset, along with "Tender," a model designed for generating diverse 2D motions from text descriptions. The framework achieves improved motion quality and text-motion alignment, demonstrating its utility for applications like human video generation and 3D motion lifting.
Key insights
- Focusing on 2D human motion generation offers a scalable and reliable alternative to 3D, capitalizing on abundant 2D video data and robust 2D pose estimation capabilities.
- Modeling whole-body movements with a part-aware attention mechanism (PA-VAE) and incorporating confidence scores (CAG) are effective for synthesizing high-fidelity 2D motions and handling real-world data complexities like occlusions.
- Training both generation and retrieval models on a significantly larger and more diverse 2D dataset improves performance, generalization, and text-motion alignment compared to existing 3D-centric approaches.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.