Reproduce “EgoSurgery-HTS: A Dataset for Egocentric Hand-Tool Segmentation in Open Surgery Videos” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nathan Darjana, Ryo Fujii, Hideo Saito, Hiroki Kajita
The Keio University research team introduced EgoSurgery-HTS, the first large-scale dataset for egocentric open-surgery video analysis, offering pixel-level annotations for 14 surgical tool types, 4 hand types, and their interactions. This resource enables the development of more accurate AI models for surgical scene understanding, effectively addressing the domain mismatch experienced by general-purpose segmentation models when applied to open surgical environments.
Key insights
- Domain-specific datasets are essential for robust performance in surgical AI; models trained on general hand/hand-object datasets exhibit severe performance degradation due to domain transfer when applied to open surgery.
- The detailed, multi-category annotations within EgoSurgery-HTS enable a more granular and holistic understanding of surgeon actions and intentions, fostering advancements in real-time surgical assistance and skill assessment.
- Current state-of-the-art instance segmentation models show encouraging, though still challenging, performance on egocentric open surgical videos, with Mask2Former generally excelling for segmentation tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.