Zero-Shot Sim2Real Transfer for Magnet-Based Tactile Sensor on Insertion Tasks

Abstract

Tactile sensing is an important sensing modality for robot manipulation. Among different types of tactile sensors, magnet-based sensors, like u-skin, balance well between tactile density, high-durability, and compactness. However, the large sim-to-real gap of tactile sensors prevents robots from acquiring useful tactile-based manipulation skills from simulation data, a recipe that has been successful for achieving complex and sophisticated control policies. Prior work has used binarization techniques to bridge the sim-to-real gap for dexterous in-hand manipulation with magnet-based sensors. However, binarization inherently loses much information that is useful in many other tasks, e.g., insertion. In our work, we propose GCS, a novel sim-to-real technique to learn contact-rich insertion skills with dense, distributed, 3-axes tactile readings from magnet-based tactile sensors. We evaluated our approach on blind insertion tasks and show successful zero-shot sim-to-real transfer of RL policies with raw tactile readings as input.