High Confidence Surgical Instrument Transparency Adopting Miniature Multi-view Endoscope System

Abstract

With the continuous advancement of minimally invasive surgery, the incisions have become increasingly smaller, leading to a proportional convergence between the physical dimensions of surgical instruments and the operational space. This phenomenon has exacerbated the issue of visual obstruction caused by the overlapping of the instruments, which now stands as a significant technical impediment to the enhancement of precision in minimally invasive procedures. Vanilla approaches rely on deep learning-based image inpainting techniques to address this issue, but their results are unreliable for surgeons making decisions. This work rethinks hardware design, sacrificing resolution to increase the view of cameras, effectively filling the instrument occluded areas through multi-view technology. Subsequently, a super-resolution method is employed to restore the details of the inpainted images. This innovative approach transforms the uncertainty of deep learning from a large-range pixel inference problem into a more controllable pixel interpolation task, thereby significantly enhancing the reliability and accuracy of the repaired results. Taking spinal endoscopic surgery as an application scenario, we designed a miniature multi-view endoscope system tailored to the specific needs of this surgery. Phantom experiments are conducted to verify the feasibility of the proposed instrument transparency method. The results demonstrate the potential of the proposed method for improving minimally invasive surgery.