Reproduce “Mixture of Disentangled Experts with Missing Modalities for Robust Multimodal Sentiment Analysis” on OpenResearch

About this paper

Xiang Li, Xiaoming Zhang, Dezhuang Miao, Xianfu Cheng, Dawei Li, Honggui Han, Zhoujun Li

Researchers from Beihang University developed the Disentangled Expert Representation Learning (DERL) framework for robust multimodal sentiment analysis, which adaptively disentangles modality-private and shared sentiment features and employs multi-level reconstruction to handle missing input modalities. The DERL framework achieved superior performance on MOSI and MOSEI datasets, demonstrating up to 2.47% higher Acc-2 and 3.90% lower regression errors compared to state-of-the-art baselines under various missing data scenarios, with competitive parameter efficiency.

Key insights

• Adaptive disentanglement, facilitated by a Mixture-of-Experts architecture, effectively handles heterogeneous information variations caused by missing modalities.
• Multi-level reconstruction across input, disentangled, and joint feature spaces provides comprehensive supervision, crucial for robustly recovering degraded multimodal semantics.
• Dynamically routing and weighting disentangled features based on their importance improves multimodal fusion by mitigating imbalances from incomplete data.

Summary by alphaXiv.