Reproduce “Mutual Information Regularization for Weakly-supervised RGB-D Salient Object Detection” on OpenResearch

About this paper

Aixuan Li, Yuxin Mao, Jing Zhang, Yuchao Dai

This work introduces a weakly-supervised RGB-D salient object detection framework that uses scribble annotations to reduce labeling costs. It employs asymmetric encoders for RGB and depth, mutual information regularization for disentangled multimodal feature learning, and a multimodal variational auto-encoder for robust prediction refinement, achieving state-of-the-art performance comparable to fully-supervised methods.

Key insights

• RGB and depth data respond differently to feature extractors, and using asymmetric backbones (ResNet50 for RGB, VGG16 for depth) empirically yields more effective and reliable feature representations.
• Explicitly minimizing mutual information between RGB and fused RGB-D features using the vCLUB estimator successfully encourages disentangled multimodal representation learning, reducing redundancy.
• A conditional multimodal VAE provides more robust and accurate prediction refinement in weakly-supervised settings by offering a stochastic, noise-aware approach compared to deterministic pseudo-labeling or post-processing methods.

Summary by alphaXiv.