Reproduce “Cosmos-Surg-dVRK: World Foundation Model-based Automated Online Evaluation of Surgical Robot Policy Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lukas Zbinden, Nigel Nelson, Juo-Tung Chen, Xinhao Chen, Ji Woong Kim, Mahdi Azizian, Axel Krieger, Sean Huver
Researchers from NVIDIA and Johns Hopkins University developed Cosmos-Surg-dVRK, a World Foundation Model-based learned simulator for surgical robot policy evaluation. The platform achieves a strong correlation (Pearson r=0.756) between automated simulated and real-world dVRK policy success rates, supporting efficient, automated assessment of complex soft tissue tasks like porcine cholecystectomy.
Key insights
- Learning soft tissue dynamics directly from surgical data distributions in a WFM circumvents the limitations of explicit physics parameterization in traditional simulators, enabling more realistic deformation modeling.
- Including 'failure' episodes in the WFM finetuning dataset is critical for learning physical constraints and avoiding physically inconsistent outcomes (hallucinations), significantly improving evaluation accuracy.
- An automated video classifier, built on V-JEPA 2, can reliably and objectively assess surgical policy outcomes in simulation, showing strong agreement with human labelers.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.