Reproduce “Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuyang Sheng, Sophia Bano, Matthew J. Clarkson, Mobarakol Islam
Surgical-DeSAM presents an architecture for automated, real-time instance segmentation of surgical instruments by decoupling the Segment Anything Model (SAM) and integrating it with a Swin-transformer-enhanced object detector for automatic bounding box prompting. This method achieved state-of-the-art performance, with Dice metrics of 89.62 on EndoVis 2017 and 90.70 on EndoVis 2018 datasets, addressing the impracticality of interactive prompting in robotic surgery.
Key insights
- Decoupling the large image encoder of a foundation model like SAM and integrating it with an automatic prompt generator enables its practical application in real-time, specialized domains.
- Utilizing a Swin-transformer backbone within the object detection component significantly enhances feature extraction and overall detection and segmentation accuracy compared to traditional CNNs.
- An end-to-end trained system can effectively combine robust object detection with precise segmentation to achieve fully automated, instance-level surgical instrument identification.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.