Cosmos-Surg-DVRK: World Foundation Model-Based Automated Online Evaluation of Surgical Robot Policy Learning

Abstract

The rise of robot-assisted surgery and vision language-action models has accelerated progress in autonomous surgical policies and efficient assessment strategies. However, evaluating these policies directly on physical robotic platforms such as the da Vinci Research Kit (dVRK) remains hindered by high costs, time demands, reproducibility challenges, and variability in execution. World foundation models (WFM) for physical AI offer a transformative approach to simulate complex real-world surgical tasks, such as soft tissue deformation, with a high degree of realism. This work introduces Cosmos-Surg dVRK, a surgical finetune of the Cosmos WFM, which, together with a trained video classifier, enables fully automated online evaluation and benchmarking of surgical policies. We evaluate Cosmos-Surg-dVRK using two independent, teleoperated, real world dVRK surgical datasets. On tabletop suture pad tasks, the automated pipeline achieves strong correlation between online rollouts in Cosmos-Surg-dVRK and policy outcomes on the real dVRK Si platform, as well as good agreement between human labelers and the V-JEPA 2-derived video classifier. Additionally, preliminary experiments using the state-of-the-art SRT-H surgical policy for ex-vivo porcine cholecystectomy tasks demonstrate promising alignment between Cosmos-Surg-dVRK and real-world dVRK evaluations, highlighting the platform's potential for more complex surgical procedures.