Reproduce “FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inference” on OpenResearch

About this paper

Jungbeom Lee, Eunji Kim, Sungmin Lee, Jangho Lee, Sungroh Yoon

FickleNet addresses the challenge of incomplete object localization in weakly supervised semantic segmentation by employing stochastic hidden unit selection during both training and inference. This approach generates more comprehensive and accurate localization maps, leading to a 2.2% mIoU improvement on the PASCAL VOC 2012 validation set over a DSRG baseline, achieving 61.2% mIoU with a VGG-16 backbone using only image-level labels.

Key insights

• Stochastic selection of hidden units during inference implicitly forces the network to associate non-discriminative object parts with discriminative ones, thereby exploring a wider range of feature combinations.
• The proposed feature map expansion significantly reduces computational time (up to 15.4x faster training, 14.2x faster CAM extraction) for stochastic inference without substantial memory increase.
• Aggregating multiple localization maps generated through stochastic inference substantially improves the robustness, coverage, and accuracy of pseudo-labels compared to single-pass methods.

Summary by alphaXiv.