Surgical tool pose estimation from monocular endoscopic videos

Abstract

Surgical tool pose estimation has been proven to be useful for high- and low- level feedback tasks including safety-enhancement, semantic feedback and surgical skill assessment. Tool pose estimation using monocular camera input is a well-studied research problem as the monocular camera is one of the ubiquitous sensor across the spectrum of robotic devices. Current state-of-the art methods for visual tool pose estimation are computationally expensive and require elaborate geometric and appearance models of surgical tools. We propose a visual tool pose estimation method that maps the visual bounding box to the 3D tool pose without any explicit knowledge of tool geometry using Gaussian process regression. The proposed approach can be generalized to any surgical tool and provides tool pose estimates with a variance estimate in real-time. We demonstrate rigorous evaluation of the method under various conditions that might effect the estimation process. In order to evaluate the algorithm, we have instrumented a standard box trainer kit with two laparoscopic tools to get simultaneous ground truth pose and a video feed.