Reproduce “Seeing Through the Tool: A Controlled Benchmark for Occlusion Robustness in Foundation Segmentation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nhan Ho, Luu Le, Thanh-Huy Nguyen, Thien Nguyen, Xiaofeng Liu, Ulas Bagci
OccSAM-Bench introduces a controlled benchmark to evaluate the robustness of foundation segmentation models against surgical tool occlusion in endoscopy, proposing a three-region evaluation protocol (visible, invisible, full mask) to provide clinically relevant performance insights. The study revealed that standard full-mask metrics can be misleading and categorized models into distinct 'Occluder-Aware' and 'Occluder-Agnostic' archetypes based on their response to hidden anatomy.
Key insights
- Standard full-mask (amodal) Dice scores are often misleading in surgical contexts, as they can obscure critical differences in model behavior and inadvertently reward clinically hazardous over-segmentation.
- Foundation models exhibit distinct behavioral archetypes: 'Occluder-Aware' models (e.g., SAM 3) prioritize visible tissue delineation and suppress predictions in occluded regions, while 'Occluder-Agnostic' models (e.g., MedSAM, MedSAM2) tend towards amodal completion.
- Domain-specific fine-tuning on clean medical datasets can compromise occlusion robustness, suggesting that current training paradigms might lead to overfitting to pristine boundaries.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.