Reproduce “TalkFlow: Structured Latent Flow Matching for Speech Driven Holistic Motion Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xinmu Wang, Fumi Honda, Xiang Gao, Xianfeng Gu, Xiyun Song, Heather Yu

TalkFlow generates speech-driven whole-body avatar motion using separate continuous tracks for upper body, lower body, and facial expression, coordinated by a speech-conditioned flow model. A graph encoder first represents whole-body structure, and flow interactions let the tracks exchange cues. On BEAT2 Speaker-2, it reports facial vertex MSE of 5.77 × 10⁻⁸, lower than MambaTalk’s 6.29 × 10⁻⁸; cross-speaker generalization is untested.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.