Reproduce “Enhancing Multimodal Emotion Recognition via Multi-Feature Encoding and Attention-Based Fusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xu Lin, Ke Wang, Hui Kang, Xinying Wang
The paper builds separate speech and facial-video representations for seven-emotion recognition, combining them through feature-level cross-modal attention. Its audio branch uses Wav2Vec2 embeddings, MFCCs, and prosodic features, while a ResNet-50-BiLSTM encodes video. On MELD, attention-based fusion reports 93.7% accuracy and 93.5% F1, versus 82.9% and 84.1% for concatenation, but controls and F1 averaging are unspecified.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.