Reproduce “Performance Evaluation of Segment Anything Model with Variational Prompting for Application to Non-Visible Spectrum Imagery” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yona Falinie A. Gaus, Neelanjan Bhowmik, Brian K. S. Isaac-Medina, Toby P. Breckon
Researchers from Durham University assessed the zero-shot segmentation performance of the Segment Anything Model (SAM) on X-ray and infrared imagery, revealing that while bounding box prompts achieved higher accuracy, point-based prompts struggled, especially in low-contrast infrared images. The study highlights the limitations of direct application of visible-light-trained foundation models to specialized non-visible spectrum domains without adaptation.
Key insights
- Bounding box prompts consistently enable SAM to achieve considerably higher segmentation accuracy and mean IoU compared to point-based prompts across all X-ray and infrared datasets evaluated.
- Point-based prompts, particularly random points, demonstrate reduced performance and struggle with precise segmentation, often leading to over-segmentation or failure to distinguish objects in challenging non-visible images.
- The substantial domain gap between SAM's visible-light training data and low-contrast infrared imagery results in particularly poor zero-shot generalization for infrared objects, indicating a severe impact of domain shift.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.