Reproduce “AdaSpeech: Adaptive Text to Speech for Custom Voice” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mingjian Chen, Xu Tan, Bohan Li, Yanqing Liu, Tao Qin, Sheng Zhao, Tie-Yan Liu
AdaSpeech, developed by Microsoft Research Asia and Microsoft Azure Speech, enables high-quality custom voice synthesis with a dramatically reduced memory footprint, addressing key challenges in commercial text-to-speech services. The system achieves voice quality comparable to methods requiring millions of parameters while using approximately 5,000 parameters per speaker by employing multi-granularity acoustic condition modeling and conditional layer normalization.
Key insights
- Capturing acoustic information at speaker, utterance, and phoneme levels addresses the challenges of diverse user-provided speech data, improving robustness to varying acoustic conditions.
- Conditional Layer Normalization (CLN) enables extreme parameter efficiency by dynamically generating speaker-specific normalization parameters, allowing only about 5,000 parameters per speaker during inference.
- The combination of multi-granularity acoustic modeling and CLN resolves the trade-off between custom voice quality and parameter footprint, making large-scale deployment economically viable.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.