Reproduce “Enhancing Multi-Modal Video Sentiment Classification Through Semi-Supervised Clustering” on OpenResearch

About this paper

Mehrshad Saadatinia, Minoo Ahmadi, Armin Abdollahi

Researchers at the University of Southern California developed a semi-supervised clustering method to improve multimodal video sentiment analysis, particularly when labeled data is scarce. Their approach leverages Deep Embedded Clustering for pre-training, achieving a 2.72 percentage point increase in accuracy on a benchmark dataset with limited labeled data compared to a baseline, and competitive performance with state-of-the-art models on the full dataset.

Key insights

• Semi-supervised clustering, specifically Deep Embedded Clustering (DEC), can effectively leverage unlabeled video data to learn more discriminative and robust feature representations.
• Pre-training with a clustering objective followed by fine-tuning on limited labeled data improves performance in multimodal sentiment analysis by organizing data in a latent space.
• Combining reconstruction, clustering, disentanglement, and supervised losses guides the model to learn independent and interpretable features, enhancing overall robustness.

Summary by alphaXiv.