Reproduce “Learn2Chat: Rethinking Dyadic Talking Heads via Interaction-Modulated Monologic Priors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zikai Huang, Siyue Chen, Xuemiao Xu, Haoxin Yang, Cheng Xu, Yihong Lin, Shengfeng He
Learn2Chat introduces a framework for generating dyadic 3D head motions by modulating pretrained monologic speech-motion priors with interaction-specific signals. The approach yields more natural and temporally coordinated conversational animations from paired audio, while achieving superior data and parameter efficiency.
Key insights
- Dyadic conversational motion can be effectively generated by modulating established monologic speech-motion priors with interaction-specific signals, rather than learning all aspects from scratch.
- Explicitly disentangling speech-driven motion semantics from interaction-induced social feedback leads to a more structured and robust framework for understanding and generating conversational behaviors.
- Incorporating dual-stream cross-attention in the audio-to-interaction prediction module is essential for capturing reciprocal acoustic dependencies and modeling inter-speaker temporal coordination.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.