Reproduce “TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yu Zhang, Wenxiang Guo, Changhao Pan, Dongyu Yao, Zhiyuan Zhu, Ziyue Jiang, Yuhan Wang, Tao Jin and 1 more
Researchers at Zhejiang University developed TCSinger 2, a multilingual zero-shot singing voice synthesis model that improves robustness to imperfect data annotations and offers flexible multi-level style control from diverse prompts. The model achieves state-of-the-art results, with an MOS-Q of 4.13 and MOS-S of 4.27 for parallel zero-shot style transfer, and an MOS-C of 4.19 for multi-level style control via text prompts.
Key insights
- Deliberately blurring phoneme and note boundaries during training makes the model robust to annotation errors and capable of generating smoother, more natural-sounding transitions.
- Contrastive learning effectively unifies diverse multi-level style information from singing, speech, and text prompts, allowing for versatile and aligned style control.
- A Mixture-of-Experts architecture, dynamically routed by language and stylistic cues, enhances both multilingual generation quality and fine-grained style modeling, improving overall control and stability.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.