Object Extrinsic Contact Surface Reconstruction through Extrinsic Contact Sensing from Visuo-tactile Measurements

Abstract

When manipulating an object, a robot must recognize not only the parts it directly grasps but also the surfaces in contact with the environment, which we refer to as extrinsic contact surfaces. These surfaces directly affect how the object interacts with its environment, and accurate surface estimation is critical for precise robotic manipulation. This study presents a novel framework for extrinsic contact surface reconstruction using vision-based tactile sensing. By leveraging marker-based tracking and analyzing kinematic constraints, we classify contact types and estimate the locations of both point and line contacts. To reconstruct the extrinsic contact surface, we compare three data integration methods: Mixed Vector Approach (MVA), Orthogonal Distance Regression (ODR), and Random Sample Consensus (RANSAC). Experimental results demonstrate that MVA achieves the highest accuracy in most cases by effectively integrating contact data while minimizing randomness. Experiments conducted on various object geometries validated the robustness of the proposed method, achieving an average positional error of 4.15 mm and an angular deviation of 4.58<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">°</sup>. The results confirm that extrinsic contact sensing enables more efficient and precise object shape estimation, providing a promising approach for robotic manipulation.