Reproduce “EMOCONV-DIFF: Diffusion-based Speech Emotion Conversion for Non-parallel and In-the-wild Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Navin Raj Prabhu, Bunlong Lay, Simon Welker, Nale Lehmann-Willenbrock, Timo Gerkmann
Researchers developed EmoConv-Diff, a diffusion-based speech emotion conversion (SEC) system that operates on non-parallel and 'in-the-wild' speech data. This system achieved state-of-the-art emotion conversion performance, particularly for extreme arousal values, while maintaining high speech quality.
Key insights
- Diffusion models demonstrate strong capabilities for disentangling speech attributes (lexical, speaker, emotion) even with non-parallel, naturalistic data.
- The inclusion of a mel spectrogram reconstruction loss, specifically when approximated for the source mel spectrogram (ˆX_0), is crucial for achieving high emotion conversion accuracy and speech quality.
- Representing emotion on a continuous arousal dimension allows for finer-grained control and more robust conversion, especially for target extreme emotional states.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.