Reproduce “MultiHuSE: A Multimodal Dataset for Humour Styles and Emotions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mary Ogbuka Kenneth, Foaad Khosmood, Abbas Edalat
MultiHuSE introduces an English multimodal dataset for recognizing four psychologically defined humour styles—affiliative, self-enhancing, aggressive, and self-deprecating—alongside neutral content and performer-reported emotions. Across 2,407 video performances of 1,463 texts, text was the strongest individual modality, while weighted multimodal fusion raised humour-style accuracy to 80.1% from 77.4% for text alone; emotion recognition remained difficult, reaching 38.6% accuracy at best.
Key insights
- Lexical content carried most of the humour-style signal: text-only classification reached 77.4% accuracy, compared with 58.5% for audio and 40.0% for visual features.
- Multimodal information was most useful for some styles rather than uniformly: affiliative-humour F1 increased from 66% with text to 74% with weighted fusion, while cross-modal attention performed best for self-enhancing humour.
- Performer-reported emotions were only moderately reproducible, with 44.0% pairwise agreement and Cohen’s kappa of 0.325, underscoring the subjectivity of these labels.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.