From Pixels to Touch: Direct Tactile Servoing With Learned Photometric Normalization

Abstract

Vision-Based Tactile Sensors (VBTSs) offer high-resolution contact information essential for robust robotic contact-rich manipulation under occlusions and variable lighting. Direct Visual Servoing (DVS) is an interesting alternative to classical Position-Based Visual Servoing (PBVS) as it operates directly on raw pixel intensities, without requiring feature extraction and simplifying the control pipeline. Applying DVS to VBTSs is a challenge because of the sensors’ strong position-dependent lighting effects that violate the brightness-constancy assumption that DVS requires. In this work, we introduce a Direct Tactile Servoing (DTS) framework that adapts the principles of DVS to VBTSs. Our approach integrates a deep convolutional U-Net that maps raw RGB tactile images to spatially normalized grayscale representations, restoring photometric consistency while preserving fine contact features. We further conduct targeted ablation studies to identify training strategies that balance generalization performance and data efficiency. This learned normalization enables the direct use of DVS control laws without explicit pose or force estimation. Experiments on a robotic manipulator equipped with a GelSight Mini sensor validate robust and reactive closed-loop tactile servoing across diverse contact conditions.