Reproduce “Homogeneous Dynamics Space for Heterogeneous Humans” on OpenResearch

About this paper

Xinpeng Liu, Junxuan Liang, Chenshuo Zhang, Zixuan Cai, Cewu Lu, Yong-Lu Li

Researchers at Shanghai Jiao Tong University developed Homogeneous Dynamics Space (HDyS), a framework designed to unify disparate human motion data and representations by learning a shared latent space. HDyS achieves superior inverse dynamics performance across multiple benchmarks and demonstrates utility in downstream applications like ground reaction force estimation and physical character control.

Key insights

• A fundamental "homogeneous essence" or motor knowledge underlies all disparate human motion data, and this knowledge can be effectively extracted by unifying heterogeneous data sources.
• Aggregating diverse datasets (including kinematics-only data like AMASS) significantly improves performance compared to training on single-source or homogeneously scaled data.
• The InfoNCE alignment loss is crucial for integrating knowledge from different modalities and representations, effectively creating a unified latent space and implicitly mitigating domain heterogeneity.
• The inverse-forward dynamics auto-encoder structure, particularly the inclusion of the Forward-Dynamics Auto-Encoder (FDAE), is vital for robust performance.

Summary by alphaXiv.