Reproduce “Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Liang Xu, Chengqun Yang, Zili Lin, Xintao Lv, Yichao Yan, Xin Jin, Zhibo Chen, Xiaokang Yang and 1 more
Researchers from Shanghai Jiao Tong University introduce Inter-X++, a comprehensive, high-fidelity benchmark for multimodal human-human interaction analysis, featuring whole-body and hand movements with rich annotations. This work also proposes OpenHHI, a unified modeling framework that achieves competitive performance across eight generative and perceptive tasks, enabling integrated understanding and generation.
Key insights
- The continuous 6D rotation representation was empirically identified as the most effective for HHI, balancing optimization stability, generation fidelity, and semantic consistency across tasks.
- Physics-based imitation policies, when fine-tuned on interaction-specific data like Inter-X++, significantly improve physical plausibility and realism of generated motions.
- A single, unified representation learned through joint optimization of motion reconstruction and semantic understanding (OpenHHI) can effectively serve both generative and perceptive HHI tasks, demonstrating knowledge transfer.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.