TacFlex: Multi-Mode Tactile Imprints Simulation for Visuotactile Sensors with Coating Patterns

Abstract

Visuotactile sensors can provide rich contact information for robots. However, how to build a high-fidelity visuotactile simulator that supports multi-mode tactile imprints and various sensor configurations remains a challenging problem. In this paper, we present TacFlex, a flexible simulator for visuotactile sensors, which physically simulates the elastomer deformation using Finite Element Methods, and focuses on linking the deformed elastomer mesh to diverse tactile imprints, including tactile images with arbitrary coating patterns and tactile 3D point clouds. We further propose a ray tracing-based rectification method to deal with multi-medium refraction effects to make the simulated tactile images more realistic. Extensive experiments are conducted to show the effectiveness of TacFlex on various sensors. Furthermore, we explore the Sim2Real performance of different tactile imprints provided by TacFlex in tactile perception and manipulation tasks, such as cylindrical object pose estimation and peg-in-hole. The perception/policy models trained in simulation are successfully deployed in real world. Finally, we discuss TacFlex's potential in robot learning.