Reproduce “SteerDuplex: Steerable Duplex Speech Dialogue Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Utkarsh Tyagi, Ramaneswaran Selvakumar, Advait Gosai, Sonal Kumar, Nikhil Barhate, Isabell Sagar, Steven Li, Miheer Bavare and 8 more

STEERDUPLEX extends a MOSHI-based 7B full-duplex speech model with explicit control over tone, persona, vocal style, accent, speaking speed, and response length. Supervised steering training raises STEERBENCH’s complete audio-steering pass rate from 20.55% for MOSHI to 65.10%, while reinforcement learning improves interruption and backchannel handling but exposes trade-offs between prompt yielding and response continuity.

Key insights

  • Spoken steerability must be evaluated across both content and delivery: individual rubric satisfaction can be much higher than complete-example success when every requested constraint must be met.
  • Reinforcement learning improves selected interaction behaviors, but optimizing interruption or promptness in isolation can produce empty outputs or reduce continuation after pauses and backchannels.
  • Targeted supervised examples substantially improve vocal and conversational control without degrading the reported broad spoken-dialogue benchmarks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.