Reproduce “GDPO-Listener: Expressive Interactive Head Generation via Auto-Regressive Flow Matching and Group reward-Decoupled Policy Optimization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhangyu Jin, Maksim Siniukov, Deuksin Kwon, Ashutosh Chaubey, Mohammad Soleymani

GDPO-Listener, a framework combining auto-regressive flow matching with Group reward-Decoupled Policy Optimization, generates expressive and contextually appropriate 3D head motions for both speakers and listeners in dyadic interactions. This approach significantly improves listener motion expressivity and naturalness compared to baselines, achieving a Listen FDD of 18.85 against 43.58 and demonstrating strong user preference in perceptual studies.

Key insights

  • Framing listener head generation as an 'alignment problem' with explicit incentives for dynamic, high-variance motions, rather than a restrictive trajectory-matching task, is crucial for overcoming the 'Regression-to-the-Mean' issue.
  • The combination of auto-regressive flow matching for robust base generation and group-decoupled reinforcement learning for post-training expressivity optimization effectively addresses the challenges of fidelity and non-deterministic variability.
  • Expanding the FLAME parameter space to include eyelid and global head rotation, alongside multimodal input conditioning (text, audio, partner motion), enables the synthesis of a richer set of nonverbal cues and explicit semantic control over listener intent.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.