Reproduce “Text-driven Emotional Style Control and Cross-speaker Style Transfer in Neural TTS” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yookyung Shin, Younggun Lee, Suhee Jo, Yeongtae Hwang, Taesu Kim
This paper from Shin et al. at Neosapience, Inc. introduces a text-to-speech system that enables emotional style control via natural language descriptions and facilitates cross-speaker style transfer across disjoint datasets. The approach creates a shared multi-modal embedding space where textual emotion descriptions are mapped to corresponding acoustic features, leading to improved naturalness and style similarity in synthesized speech.
Key insights
- Creating a shared multi-modal embedding space allows seamless translation between natural language emotion descriptions and acoustic style features.
- The style contrastive loss, inspired by CLIP, is critical for learning effective text-to-style mappings and generalizing to unseen styles.
- The system successfully handles disjoint datasets, enabling emotional style transfer to speakers who never recorded those specific emotions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.