Reproduce “Fine-Grained Quantitative Emotion Editing for Speech Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sho Inoue, Kun Zhou, Shuai Wang, Haizhou Li
Researchers from CUHK-Shenzhen and Alibaba Group developed a Hierarchical Emotion Distribution (HED) framework for text-to-speech generation, enabling quantitative, fine-grained control and editing of emotional intensity at phoneme, word, and utterance levels. The approach integrates an automated emotion intensity extractor into a FastSpeech2 model, achieving superior speech quality (MOS 3.596 vs. 3.544) and emotional expressiveness compared to baseline models.
Key insights
- Speech emotion can be effectively modeled and controlled by representing its intensity quantitatively across a hierarchical structure, including phoneme, word, and utterance levels.
- An automated system for quantifying emotion intensity using relative attributes can provide 'soft labels' that enable precise, fine-grained emotion control in neural TTS models without extensive manual annotation.
- Integrating hierarchical and quantitative emotion parameters into the variance adaptor of a FastSpeech2 architecture allows for robust emotional expressiveness and direct user-controlled editing during speech generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.