Reproduce “AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yao Shi, Hui Bu, Xin Xu, Shaoji Zhang, Ming Li

The AISHELL-3 foundation introduces AISHELL-3, an 85-hour open-source multi-speaker Mandarin speech corpus from 218 native speakers, meticulously annotated for linguistic complexities. A strong multi-speaker Tacotron-2 baseline system, enhanced with a feedback constraint on speaker embeddings, achieves an average cosine similarity of 0.868 for unseen speakers and an SV-EER of 9.46% on the test set, demonstrating robust zero-shot voice cloning capabilities.

Key insights

  • Manual pinyin transcription, carefully accounting for homographs, tone sandhi, and erization, is crucial for creating high-quality Mandarin speech corpora for TTS research.
  • A feedback constraint, which minimizes the cosine distance between ground-truth and synthesized speaker embeddings, significantly enhances speaker similarity and zero-shot voice cloning performance in multi-speaker TTS.
  • Strategic data augmentation and preprocessing, such as silence trimming and concatenating sentences, improve the stability of attention mechanisms and naturalness of prosody in neural TTS models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.