Reproduce “Amodal SAM: A Unified Amodal Segmentation Framework with Generalization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Bo Zhang, Zhuotao Tian, Xin Tao, Songlin Tang, Jun Yu, Wenjie Pei

Amodal SAM is a unified framework extending the Segment Anything Model (SAM) to perform amodal segmentation for both images and videos, demonstrating strong generalization in open-world scenarios. The framework achieves state-of-the-art performance on various amodal benchmarks, significantly improving the prediction of occluded regions and reducing reliance on manual amodal annotations through synthetic data generation.

Key insights

  • Encoder-focused adaptation is more effective than decoder adaptation for extending SAM's generalization capabilities to amodal segmentation tasks, as it better bridges the domain gap for perceiving occluded regions.
  • The Spatial Completion Adapter (SCA), utilizing a gated convolution, provides a dynamic feature selection mechanism crucial for accurately reconstructing occluded geometries within SAM's framework.
  • The combination of automated synthetic data generation and a specialized multi-component loss function effectively guides a foundation model to perform complex "occlusion-aware hallucination" for amodal understanding.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.