Reproduce “Qwen-Audio-3.1-Realtime: Towards Reliable Agentic Voice Interaction” on OpenResearch

About this paper

Lujia Bao, Qian Chen, Luyao Cheng, Chong Deng, Yuxiang Kong, Xiangang Li, Xu Li, Jiaqing Liu and 10 more

Qwen-Audio-3.1-Realtime trains voice interaction as coordinated action, conversation, and timing. Executable environments combine tools, state, policies, scripted users, and checks of terminal state, authorized writes, and dialogue behavior, while a conversational policy selects whether to listen, speak, stop, or resume. It reaches 82.0% versus 78.4% on a half-duplex τ-Voice adaptation and reduces background-speech responses from 73.0% to 13.0% on FDB v1.5, but these descriptive comparisons do not establish causal effects or broad real-world reliability.

Key insights

• Reliable voice agency depends on joint control of reasoning, execution, and interaction timing rather than optimizing speech recognition or response latency in isolation.
• Executable environments provide stronger supervision than surface-level tool-call matching because they can detect unauthorized writes, incorrect database states, unsupported claims, and failures to verify outcomes.
• The model became more selective in full-duplex interaction: responses to background or other-directed speech decreased sharply while appropriate resumption improved, suggesting better control over when not to speak.

Summary by alphaXiv.