Grasp Independent Indirect Tool Force Estimation Using Vision-Based Tactile Sensors

Abstract

Humans possess the capability to seamlessly integrate tools into their body schema, enabling precise and adaptive interactions with the environment. This touch-mediated ability allows us to dexterously use tools in everyday tasks, an ability currently lacking in robotic systems. In this work, we propose a novel method for indirect force estimation in robotic tool use, a prerequisite for advanced tool use, leveraging vision-based tactile sensing (VTS) and deep learning techniques. By capturing high-resolution spatial deformations from tactile images, our model implicitly infers force transmission dynamics without requiring explicit knowledge of tool properties or material characteristics. We validate our approach across multiple tool types using a single trained machine learning model, demonstrating its generalization capability. This work represents the first demonstration of indirect force estimation for tool-mediated robotic interactions, offering a pathway toward more dexterous and adaptive robotic tool use in real-world applications.