Reproduce “ECHO: Dyadic 3D Facial Motion Generation with Asymmetric Deterministic Articulation and Stochastic Reaction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhuoqiang Cai, Yujie Sun, Chaoyue Niu, Hongyun Yu, Zhiwen Chen, Chengfei Lv, Fan Wu
ECHO introduces an asymmetric framework for dyadic 3D facial motion generation, capable of fluidly transitioning between stable, speech-aligned articulation and diverse, realistic listener reactions using only dual-stream audio. The system reduces Fréchet Distance by 62.7% and increases motion diversity by 73.5% compared to prior stochastic baselines, while achieving the highest perceptual scores in user studies.
Key insights
- Conversational facial motion is fundamentally asymmetric, requiring distinct modeling approaches for speech-driven articulation and weakly constrained, diverse listener reactions.
- Decomposing motion into a deterministic anchor and a stochastic residual effectively balances articulatory fidelity with interaction diversity, overcoming limitations of uniform generative strategies.
- Targeted, annealed stochasticity (applied early in the generation process and decaying) allows for realistic listener motion modes without compromising the stability of speech-critical movements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.