Reproduce “Unsupervised Structural Scene Decomposition via Foreground-Aware Slot Attention with Pseudo-Mask Guidance” on OpenResearch

About this paper

Huankun Sheng, Ming Li, Yixiang Wei, Yeying Fan, Yu-Hui Wen, Tieliang Gong, Yong-Jin Liu

FASA organizes unsupervised scene decomposition hierarchically: it first separates foreground from background using clustered self-supervised ViT features, then assigns the foreground to object slots guided by MaskCut pseudo-masks. It improves instance-level segmentation across synthetic and natural-image benchmarks, reaching 48.2 mIoU on MOVi-C, 34.1 on COCO, and 50.2 on PASCAL VOC, while remaining vulnerable to crowded scenes, small objects, and reliance on fixed slots and a corner-based foreground prior.

Key insights

• Separating foreground from background before instance discovery reduces competition between contextual regions and object slots; the dedicated module reaches foreground–background mBO scores of 60.1 on VOC and 51.8 on COCO.
• Pseudo-mask guidance is central to preserving object integrity: removing the BCE alignment loss on COCO lowers mIoU from 34.1 to 25.2 and instance mBO from 36.5 to 26.8.
• The approach gains spatial and transfer benefits without language supervision, but its assumptions limit robustness: fixed slot counts, a corner-based foreground prior, imperfect MaskCut masks, and dependence on DINOv2 features remain important constraints.

Summary by alphaXiv.