Reproduce “Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuehao Wang, Yonghao Long, Siu Hin Fan, Qi Dou
A neural rendering framework reconstructs highly deformable soft tissues in robotic surgery videos, employing dynamic neural radiance fields, tool mask-guided ray casting, and stereo depth-cueing ray marching. The method achieves improved rendering quality with a PSNR of 29.831, SSIM of 0.925, and LPIPS of 0.081, outperforming previous approaches in handling large deformations and tool occlusions.
Key insights
- Dynamic neural radiance fields can effectively model complex, large-scale non-rigid deformations and topology changes (e.g., cutting) in surgical tissues.
- Explicitly integrating tool masks into the ray casting process, along with a mechanism to balance sampling rates of occluded regions, significantly improves reconstruction completeness despite instrument obstruction.
- Incorporating coarse stereo depth priors through guided ray marching and statistically refined depth supervision enhances 3D reconstruction quality, compensating for sparse viewpoint data in endoscopic videos.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.