Reproduce “MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, Rada Mihalcea

MELD, a multimodal multi-party dataset for emotion recognition in conversations, provides approximately 13,000 utterances from the *Friends* TV series, complete with synchronized text, audio, and video. Researchers demonstrated that integrating both multimodal information and conversational context enhances the accuracy of emotion and sentiment classification in dialogues.

Key insights

  • Multimodal context significantly improves annotation consistency and accuracy for emotions in conversations, as demonstrated by MELD's higher Fleiss' kappa.
  • Conversational context, particularly modeling inter-speaker dependencies and emotion shifts, is crucial for accurate emotion recognition in multi-party settings, yielding performance improvements over non-contextual approaches.
  • While textual modality is the primary contributor to emotion and sentiment expression, multimodal fusion (text and audio) consistently improves recognition performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.