Reproduce “Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yang Liu, Muzhi Zhu, Hengtao Li, Hao Chen, Xinlong Wang, Chunhua Shen

Matcher is a training-free perception framework that enables diverse segmentation tasks using off-the-shelf vision foundation models and a single in-context example. The system achieved state-of-the-art performance in few-shot semantic segmentation, improving mIoU by 1.6% on COCO-20i and 14.4% on LVIS-92i compared to previous methods, and surpassed benchmarks in one-shot object part segmentation.

Key insights

  • Diverse perception tasks can be solved by intelligently orchestrating existing, off-the-shelf vision foundation models in a training-free manner.
  • Combining a robust feature extractor (like DINOv2) with a class-agnostic segmenter (like SAM) creates a synergy that imbues the segmenter with semantic understanding.
  • Bidirectional matching and instance-level mask refinement using structural (Earth Mover's Distance) and contextual metrics significantly improve the accuracy and semantic relevance of segmentation outputs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.