Force-Centric Bidirectional Cross-Domain Interpretation Between Soft-Body Simulation and Piezoresistive E-Skin

Abstract

Cutaneous tactile feedback is essential for robotic manipulation, particularly when interacting with deformable objects with limited visual sensing. Learning-based approaches are increasingly adopted but rely on large volumes of tactile data. To meet this demand, Sim-to-Real strategies are widely used to generate synthetic high-resolution tactile images. However, in many tactile servo tasks, force information is more critical than the tactile image. Electronic skins (e-skins) offer direct force sensing, but remain underutilized in Sim-to-Real approaches due to the difficulty of simulating their output signals. Force-centric interpretations across real and simulation domains can facilitate the application of learning-based methods to e-skin sensing. We propose a framework that combines a soft-body simulator with a 10×10 piezoresistive e-skin. A shared feature encoder and dual-headed decoder enable bidirectional interpretation between e-skin normal force mapping readings and simulation normal stress fields. This framework supports both high-resolution stress field reconstruction and virtual tactile sensing data generation. Experiments show that the framework reduces the spatial discrepancy between real and simulated data, enhances multi-frame gradient consistency, and reconstructs edge and feature details, achieving an SSIM of 0.79 in cross-domain interpretation.