Graph-Based Spatial Reasoning for Tracking Landmarks in Dynamic Laparoscopic Environments

Abstract

Accurate anatomical landmark tracking is crucial yet challenging in laparoscopic surgery due to the changing appearance of landmarks during dynamic tool-anatomy interactions and visual domain shifts between cases. Unlike appearance-based detection methods, this work proposes a novel graph-based approach to reconstruct the entire target landmark area by explicitly modeling the evolving spatial relations over time among scenario entities, including observable regions, surgical tools, and landmarks. Considering tool-anatomy interactions, we present the Tool-Anatomy Interaction Graph (TAI-G), a spatio-temporal graph that captures spatial dependencies among entities, attribute interactions within entities, and temporal dependencies of spatial relations. To mitigate domain shifts, geometric segmentation features are designated as node attributes, representing domain-invariant image information in the graph space. Message passing with attention helps propagate information across TAI-G, enhancing robust tracking by reconstructing landmark data. Evaluated on laparoscopic cholecystectomy, our framework demonstrates effective handling of complex tool-anatomy interactions and visual domain gaps to accurately track landmarks, showing promise in enhancing the stability and reliability of intricate surgical tasks.