Fast Graph Refinement and Implicit Neural Representation for Tissue Tracking

Abstract

Tracking of tissue in the surgical environment is often done via locating frame-to-frame keypoint correspondences, and then using these correspondences to warp a prior underlying model such as a spline, mesh, or embedded deformation. We introduce a novel learned model which takes keypoint correspondences as input and enables a prior-free estimation of deformation at any location. For fast point tracking, our model allows for sparse queries, unlike dense grid based CNNs, which run on full images. Our model begins with a novel graph-based point refinement scheme which refines matched keypoints, updating their features and movement instead of discarding possible outliers. Then, we use these refined matches to learn a novel neural implicit representation for estimating movement of any location given its k-nearest neighbor (k-NN) keypoints. We name our implicit deformation model KINFlow (k-NN implicit neural flow). We demonstrate the performance of KINFlow photometrically on three different datasets. KINFlow is the first model to use a graph network to estimate flow of arbitrary query points, and can estimate movement of 1024 points in under 3 ms.