Reproduce “Borderless Long Speech Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xingchen Song, Di Wu, Dinghao Zhou, Pengyu Cheng, Hongwu Ding, Yunchao He, Jie Wang, Shengfan Shen and 7 more
The Any2Speech framework facilitates borderless long speech synthesis by overcoming limitations of prior generations, enabling global coherence across extended contexts and full acoustic scene modeling. It achieves unified multi-task capabilities, including VoiceDesigner and multi-speaker synthesis, alongside improved instruction following and expressiveness through novel data strategies and an agentic architecture.
Key insights
- Embracing and richly annotating 'noisy', complex real-world audio data is crucial for training models capable of borderless, expressive, and acoustically scene-aware speech synthesis.
- A hierarchical, natural-language-captioned annotation schema can transform narrow text inputs into a comprehensive, multi-dimensional control protocol, formalizing information flow for expressive audio generation.
- Adopting a two-stage 'understand first, synthesize second' approach (Chain-of-Thought) for prosody planning enhances precision, global coherence, and interpretability in complex expressive speech.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.