TacTip-Based Dynamic Contact Force Estimation with Sequential Tactile Images and Its Applications to Robotic Force Tracking

Abstract

Force estimation is crucial for robotics, human--machine interaction, and industrial automation. However, traditional methods are often hindered by high cost, mechanical wear, and limited accuracy in dynamic scenarios. Vision-based tactile sensing provides a promising alternative, yet existing approaches commonly rely on static calibration and degrade under dynamic interactions such as slip. To overcome these limitations, we present a novel force prediction framework for TacTip sensors, termed as Frame-stack Force Prediction Method (FFPM). The framework integrates a Dynamic Tactile Flow Encoder to capture spatiotemporal features, enabling accurate modeling of dynamic force variations. An Exponentially Weighted Residual Correction strategy is further introduced to refine predictions by leveraging historical residuals, yielding smoother and more reliable force estimation. The predicted forces are incorporated into a force-tracking impedance control scheme, achieving precise tracking during slip interactions. Experiments on our constructed dataset demonstrate state-of-the-art performance, reducing MAPE to 12.54%, and further validate the effectiveness of the proposed framework in real-world dynamic force estimation and control.