Reproduce “Vision-Based Neurosurgical Guidance: Unsupervised Localization and Camera-Pose Prediction” on OpenResearch

About this paper

Gary Sarwin, Alessandro Carretta, Victor Staartjes, Matteo Zoli, Diego Mazzatenta, Luca Regli, Carlo Serra, Ender Konukoglu

An unsupervised deep learning method learns to localize anatomical structures and predict camera viewing angles from endoscopic neurosurgical videos. This approach enhanced the coherence of the learned surgical path representation and approximated pitch/yaw angles with mean errors of 0.43/0.69 degrees on a synthetic dataset.

Key insights

• Explicitly modeling camera viewing angles (pitch and yaw) in an unsupervised latent space significantly improves the coherence and accuracy of the learned surgical path.
• Utilizing semantic bounding box detections of anatomical structures provides a more robust input for vision-based guidance in challenging endoscopic environments than traditional feature-based methods.
• A transformer-based Autoencoder can effectively disentangle positional information from viewing orientation, leading to a more interpretable representation of the surgical environment.

Summary by alphaXiv.