MonoIR: Inpainting and Reconstruction for Monocular Endoscope Deformation Scenes

Abstract

Monocular endoscopic scene reconstruction is challenging due to limited viewpoints and interference from surgical instruments. While 3D Gaussian-based methods are popular for their strong reconstruction capabilities and efficiency, they often rely on sensors or stereo depth, resulting in blurred tissue areas when instruments obstruct the view. To overcome these issues, we propose MonoIR, an inpainting and reconstruction method that produces spatiotemporally consistent videos and high-quality reconstructions. Our approach employs a propagation-based method to inpaint video holes, enhanced by optical flow constraints for robustness and an additional transformer-based module to address detail loss. We also introduce the normal and depth regularization with confidence to improve reconstruction quality. Extensive experiments demonstrate that MonoIR efficiently reconstructs deformed tissues and outperforms both monocular and binocular methods in key metrics.