Reproduce “Bridge the Points: Graph-based Few-shot Segment Anything Semantically” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Anqi Zhang, Guangyu Gao, Jianbo Jiao, Chi Harold Liu, Yunchao Wei

This research presents a graph-based framework for few-shot semantic segmentation, leveraging the Segment Anything Model (SAM) in an automated, training-free, and hyperparameter-independent manner. The approach achieves state-of-the-art mean Intersection over Union (mIoU) across various datasets, including 72.1% for 1-shot on Pascal-5^i and a 5.8% mIoU improvement over previous methods on COCO-20^i.

Key insights

  • Explicitly modeling the coverage relationships between fine-grained point prompts and coarse-grained SAM masks using a directed graph provides an effective and hyperparameter-free method for clustering masks into semantically coherent components.
  • Leveraging a combination of positive (foreground) and negative (background) contextual information for dynamic point selection, along with self-consistency checks for mask refinement, is crucial for filtering ambiguous SAM masks and false positives.
  • A training-free pipeline that utilizes one-time SAM mask generation significantly improves efficiency and automation in few-shot semantic segmentation, offering a more practical alternative to iterative or hyperparameter-dependent methods.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.