Reproduce “Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Weiyao Wang, Matt Feiszli, Heng Wang, Jitendra Malik, Du Tran

Meta AI Research introduces Generic Grouping Networks (GGNs) for open-world instance segmentation, enabling models to segment objects from both seen and unseen categories by exploiting pseudo ground truth generated from learned pixel-level pairwise affinities. This approach significantly improves average recall on unseen classes and outperforms baselines on real-world datasets like ADE20K and UVO.

Key insights

  • Pixel-level pairwise affinity prediction generalizes effectively to capture semantic boundaries of objects from unseen categories, a crucial aspect for open-world segmentation.
  • Ultrametric Contour Map (UCM)-based grouping algorithms, particularly OWT+gPb+UCM, are highly effective in leveraging predicted pairwise affinities to assemble coherent object instances.
  • Training an instance segmentation network directly on a diverse set of pseudo-ground-truth masks, in addition to human annotations, enables significant improvements in open-world generalization and performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.