Reproduce “DAiSEE: Towards User Engagement Recognition in the Wild” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Abhay Gupta, Arjun D'Cunha, Kamal Awasthi, Vineeth Balasubramanian

Researchers from Microsoft, IIT Hyderabad, and IIIT Vadodara present DAiSEE, a publicly available video dataset designed for recognizing user engagement, boredom, confusion, and frustration in naturalistic e-learning environments, with each state labeled by four intensity levels. Benchmark experiments on DAiSEE show temporal deep learning models like LRCN achieve superior performance for states such as confusion (72.3% accuracy) and engagement (57.9% accuracy) compared to static approaches.

Key insights

  • Temporal deep learning models (C3D, LRCN) generally outperform static, frame-based models for recognizing subtle, prolonged affective states, indicating the importance of dynamic cues.
  • Distinguishing between four granular intensity levels for each affective state poses a much harder problem than binary classification, highlighting the inherent complexity of fine-grained affect recognition.
  • Real-world challenges such as varying illumination, facial occlusions, and non-frontal poses significantly impact the accuracy of "in the wild" affect recognition models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.