Reproduce “NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xu Tan, Jiawei Chen, Haohe Liu, Jian Cong, Chen Zhang, Yanqing Liu, Xi Wang, Yichong Leng and 6 more
Microsoft Research Asia presents NaturalSpeech, an end-to-end text-to-waveform synthesis system that achieves speech quality statistically indistinguishable from human recordings on the LJSpeech dataset. This system is the first to demonstrate such a result, validated by Wilcoxon signed rank tests on Comparative Mean Opinion Scores.
Key insights
- Human-level TTS quality can be rigorously defined and verified by demonstrating no statistically significant difference from human speech, as confirmed by p-values greater than 0.05 in Wilcoxon signed rank tests on perception scores.
- Addressing the core challenges of training-inference mismatch, the one-to-many mapping problem, and representation capacity through targeted architectural innovations is essential for pushing TTS quality boundaries.
- Employing a memory mechanism within the VAE effectively reduces the information burden on the latent representations, simplifying the posterior and facilitating a better match with the text-derived prior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.