Reproduce “InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarios” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yinghao Huang, Leo Ho, Dafei Qin, Mingyi Shi, Taku Komura
Researchers from The University of Hong Kong introduced InterAct, a multi-modal dataset and diffusion model framework for generating realistic, interactive body motions and facial expressions for two individuals from audio input. The system achieves lower FIDr and higher diversity in multi-person interactions and competitive facial animation metrics, conditioned on spatial relations and action labels.
Key insights
- Relative spatial positions (e.g., 'facing' vs. 'not-facing') and inter-personal relationship types significantly influence the diversity and characteristics of both body motion and facial expressions during two-person interactions.
- Conditioning motion generation on contextual factors such as relative position, relationship, and emotional state is essential for achieving realistic and contextually appropriate multi-person interactive animations.
- Diffusion models, when equipped with appropriate multi-modal and interactive conditioning (e.g., concatenated inputs/outputs and relative spatial offsets), can effectively synthesize complex, two-person interactive behaviors.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.