Vision-Based Haptic Rendering With Self-Occlusion Resilience Using Shadow Correspondence

Abstract

Vision-based haptic feedback provides cost-effective preemptive protection and real-time guidance, enhancing teleoperation with reduced system complexity. However, challenges arise as the instrument approaches target object, leading to occlusion of the point cloud behind the remote instrument, known as the self-occlusion issue. Prior solutions relying on historical point clouds or multiple viewpoints to refill the occluded region encounter adaptability issues for prolonged occlusion and limited space, thus hindering practical implementation. This letter introduces a novel non-refilling-based method for haptic force rendering, leveraging the correspondence between the tool-tip position and the tip position of the shadow-like occluded region. Experimental results demonstrate the proposed method's resilience across self-occlusion and dynamic environments, highlighting its practical applicability in robotic teleoperation.