Open-Set Tactile Recognition Using Regression of Mechanical Properties

Abstract

Tactile exploration enables robots to acquire rich mechanical information about objects through physical interaction and enhances their perception and manipulation in unstructured environments. However, existing tactile object recognition methods are limited by closed-set assumptions, recognizing only predefined categories and relying heavily on labeled data. This limits their ability to handle unseen objects. Here we propose an open-set tactile recognition approach that broadens the ability of robots to identify known and novel objects. The approach leverages the intrinsic mechanical properties of objects, estimated online through haptic interaction. It integrates supervised learning for recognizing known objects with unsupervised clustering for characterizing novel ones. The inclusion of new objects is enabled by regression-based mechanical property estimation combined with distance-based online clustering. This allows robots to generalize effectively and extract reliable features from unseen objects, improving perception in dynamic environments. Validation on a 20 object dataset with diverse mechanical characteristics shows the efficacy of the approach, achieving a 96.02 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 1.69% recognition rate for 12 known objects and detecting 8 novel objects with a 90.79 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 5.45% accuracy. After integrating the newly identified objects into the robot’s knowledge base, the approach achieves an Adjusted Rand Index of 0.701 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 0.096, confirming effective clustering. These results show the potential of the proposed method to advance open-set tactile perception and support more adaptive robot interaction in real-world settings.