Reproduce “SAM-REF: Introducing Image-Prompt Synergy during Interaction for Detail Enhancement in the Segment Anything Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chongkai Yu, Ting Liu, Anqi Li, Xiaochao Qu, Chengjing Wu, Luoqi Liu, Xiaolin Hu
SAM-REF enhances the detail extraction capabilities of large interactive segmentation models like the Segment Anything Model (SAM) by introducing image-prompt synergy during a lightweight, two-stage refinement process. This framework achieves state-of-the-art accuracy across multiple benchmarks while maintaining SAM's high efficiency.
Key insights
- Reintroducing image-prompt synergy in a lightweight, decoupled refinement stage effectively combines the detail-capturing strengths of early fusion with the efficiency of late fusion.
- A coarse-to-fine iterative refinement strategy, employing both global and local fusion refiners, progressively addresses detail deficiencies in SAM's output.
- The Dynamic Selector adaptively manages the refinement scope, preventing degradation of already accurate areas and ensuring efficient convergence by choosing the most appropriate paste target for local refinements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.