Reproduce “XiaoiceSing: A High-Quality and Integrated Singing Voice Synthesis System” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Peiling Lu, Jie Wu, Jian Luan, Xu Tan, Li Zhou
Researchers from Microsoft's Xiaoice team and Microsoft Research Asia developed XiaoiceSing, an integrated neural network system for singing voice synthesis based on a modified FastSpeech architecture. The system significantly improves musical accuracy and expressive dynamics, achieving higher subjective scores for sound quality and naturalness, alongside reduced F0 and duration prediction errors compared to prior methods.
Key insights
- A residual F0 connection improves pitch accuracy by having the decoder predict only the deviation from the target note pitch, rather than absolute F0 values.
- Incorporating a syllable-level duration loss in addition to phoneme duration loss leads to more natural rhythmic timing and reduced error accumulation in synthesized singing.
- Joint training of all components—spectrum, F0, and duration models—within a unified neural architecture ensures consistency and captures inter-dependencies between musical features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.