Reproduce “EmoQ: Speech Emotion Recognition via Speech-Aware Q-Former and Large Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiqing Yang, Man-Wai Mak
EMOQ introduces a multimodal large language model framework for Speech Emotion Recognition, employing a speech-aware Q-Former for cross-modal feature fusion and a multi-objective learning strategy. This approach consistently achieved state-of-the-art accuracy on the IEMOCAP and MELD datasets.
Key insights
- A dedicated EmoQ-Former effectively bridges acoustic and textual modalities, generating a discriminative multimodal representation that mitigates cross-modal alignment difficulties.
- The Multi-objective Affective Learning framework, integrating contrastive and focal losses, significantly enhances the discriminative power of emotional embeddings and manages class imbalance effectively.
- Injecting refined multimodal features via soft-prompts into an LLM allows it to fully leverage its language understanding while integrating rich emotional cues for improved emotion recognition.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.