Reproduce “FindingEmo: An Image Dataset for Emotion Recognition in the Wild” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Laurent Mertens, Elahe' Yargholi, Hans Op de Beeck, Jan Van den Stock, Joost Vennekens

Researchers from KU Leuven introduced FindingEmo, an image dataset featuring 25,869 naturalistic scenes with multiple people, designed to advance emotion recognition towards understanding higher-order social cognition. Baseline evaluations demonstrate the dataset's difficulty for current models, achieving a maximum Weighted F1 of 0.44 for Emo8 classification and MAE above 1.0 for Valence/Arousal regression, with older CNNs being competitive for classification and ViTs excelling in regression.

Key insights

  • Human annotators often disagree on discrete emotion labels for complex scenes but tend to select 'nearby' emotions on Plutchik's Wheel, reflecting the inherent ambiguity and nuance of human emotion perception.
  • Object-centric features, typically learned from ImageNet, prove more effective than scene-centric features (from Places365) for recognizing emotions in social contexts on FindingEmo.
  • The emotion recognition tasks on FindingEmo are challenging for state-of-the-art deep learning models, suggesting a need for more advanced approaches beyond current architectures.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.