Reproduce “YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Edresson Casanova, Julian Weber, Christopher Shulby, Arnaldo Candido Junior, Eren Gölge, Moacir Antonelli Ponti
YourTTS introduces a multilingual zero-shot multi-speaker text-to-speech (TTS) and voice conversion system, built upon the VITS architecture, which achieves state-of-the-art performance on English benchmarks while demonstrating effective synthesis and rapid adaptation for low-resource languages using minimal target speaker data.
Key insights
- A multilingual training paradigm, even with limited target language data, can facilitate high-quality zero-shot multi-speaker TTS and voice conversion for low-resource languages.
- Rapid speaker adaptation is highly effective, significantly boosting voice similarity for unseen speakers with as little as 20 seconds of fine-tuning data.
- Leveraging robust pre-trained speaker encoders within an end-to-end VAE architecture like VITS provides a strong foundation for achieving state-of-the-art zero-shot capabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.