Reproduce “Revealing Emotional Clusters in Speaker Embeddings: A Contrastive Learning Strategy for Speech Emotion Recognition” on OpenResearch

About this paper

Ismail Rasim Ulgen, Zongyang Du, Carlos Busso, Berrak Sisman

Researchers at The University of Texas at Dallas uncovered that state-of-the-art speaker embeddings directly contain emotional information, which they harnessed through a new utterance-level, unsupervised contrastive learning strategy to boost Speech Emotion Recognition (SER) performance. Their proposed multi-task learning framework, integrating contrastive learning with speaker classification, achieved a Unweighted Average Recall (UAR) of 73.80% on IEMOCAP and 83.01% on CREMA-D.

Key insights

• State-of-the-art deep speaker embeddings directly encode discernible emotion-related information, manifesting as consistent intra-speaker clusters for different emotional states.
• An unsupervised contrastive learning objective can effectively exploit these inherent emotion clusters within speaker embeddings, even without explicit emotion labels during pre-training.
• Combining emotion-centric contrastive learning with speaker classification in a multi-task framework provides a synergistic approach, demonstrating that speaker and emotion recognition are inherently linked tasks.

Summary by alphaXiv.