Reproduce “Visualising Model Training via Vowel Space for Text-To-Speech Systems” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Binu Abeysinghe, Jesin James, Catherine I. Watson, Felix Marattukalam
Researchers introduce an intermediary evaluation method for text-to-speech models, visualizing training progress by tracking vowel space changes. This approach provides an acoustic-based diagnostic during development, demonstrating the synthetic speech's accent shift from General American English to New Zealand English and offering insights for model optimization and data curation.
Key insights
- The vowel space of a speech synthesis model dynamically transforms during training, progressively shifting its acoustic characteristics towards the target language or accent.
- The learning process for vowel space transformation is non-linear, with the most significant and rapid changes occurring within the initial phase of fine-tuning.
- Human listeners perceive changes in synthetic speech accent that directly correlate with the observed shifts and convergence of the model's vowel space.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.