Reproduce “Multimodal Information Interactions Bottleneck for Multimodal Self-Supervised Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Liangjian Wen, Jiangtao Zheng, Jiang Duan, Qun Dai, Yong Dai, Jianzhuang Liu, Jun Wang, Shu-Dong Huang and 3 more
MIIB learns modality-specific and fused representations alongside a separate stream intended to capture information available through combining modalities. Its entropy-based objective is a surrogate, not a literal channel-capacity constraint or guarantee of recovering theoretical synergy. On a constructed image-pair synergy task, MIIB reaches 79.1% accuracy versus CoMM’s 71.4%; on MM-IMDb linear probing, it scores 56.20% Macro-F1 versus 53.79% with the same pretrained CLIP backbone.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.