Deep Coarse-to-Fine Networks for Robust Segmentation and Pose Estimation of Surgical Suturing Threads

Abstract

Autonomous suturing is a critical challenge in robot-assisted surgery, where accurate segmentation and pose estimation of suturing threads are essential prerequisites. However, suturing threads are easily occluded by moving instruments and embedded in deformable tissues which make the task much more challenging. To address this, we propose a coarse-to-fine network for detailed segmentation and pose estimation of suturing threads. The coarse stage aims to capture global thread structure, while the fine stage refines the detailed structure through error residual correction. A spatial context fusion module is incorporated to improve the perception of occluded regions, and weighted balanced cross entropy loss as well as hard sample mining strategy is implemented to enhance small target segmentation performance. To deal with severe occlusions, topological constraints are utilized to effectively identify and reconstruct invisible thread segments. Experiments have been conducted on three datasets collected from different surgical scenes including phantom, endoscopy, and microsurgery. Both quantitative and qualitative results have demonstrated that our proposed framework outperforms baseline methods on segmentation and pose estimation of suturing threads, particularly in detecting occluded threads. Our proposed framework generalizes well across different surgical scenarios, showing its potential for automatic suturing.