Reproduce “Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuxuan Wang, Daisy Stanton, Yu Zhang, RJ Skerry-Ryan, Eric Battenberg, Joel Shor, Ying Xiao, Fei Ren and 2 more
Google researchers developed Global Style Tokens (GSTs) for end-to-end speech synthesis, enabling unsupervised modeling, fine-grained control, and transfer of speaking styles from arbitrary audio. The system demonstrates strong robustness to noisy, unlabelled data and successfully disentangles style, noise, and speaker identity.
Key insights
- Global Style Tokens can successfully learn and disentangle interpretable stylistic attributes (e.g., pitch contour, speaking rate, animation) from raw speech data without any explicit supervision.
- The learned tokens offer flexible and interpretable control over synthesis styles, allowing for direct selection, scaling, blending, and even piecewise style changes within an utterance.
- The architecture enables robust non-parallel style transfer from arbitrary reference audio clips and can effectively factorize out noise and speaker identity from stylistic information present in diverse, real-world data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.