Reproduce “Seeing Through Smoke: Surgical Desmoking for Improved Visual Perception” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jingpei Lu, Fengyi Jiang, Xiaorui Zhang, Lingbo Jin, Omid Mohareri
Researchers at Intuitive Surgical developed a transformer-based digital desmoking model and a large-scale synthetic data generation pipeline, alongside curating the largest known paired surgical smoke dataset. The model improved image reconstruction quality and instrument segmentation performance in endoscopic videos.
Key insights
- A combination of synthetic data generation and fine-tuning on a relatively smaller real-world dataset effectively trains robust desmoking models, addressing data scarcity in medical imaging.
- The transformer-based architecture with a physics-inspired head effectively reconstructs smoke-free images and predicts smoke maps in real-time, maintaining color fidelity and reducing artifacts.
- While desmoking improves specific downstream tasks like instrument segmentation, it does not universally enhance all computer vision tasks (e.g., stereo depth estimation), suggesting some foundational models may be inherently robust to smoke or desmoking might alter image statistics in ways that impact geometry-based cues.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.