Self-Supervised Complementary Learning between Vision and Tactility by Probing Action into an Open-Mouth Container
Daiki Takamori, Tomohiro Hayakawa, Yuichi Kobayashi, Kosuke Hara, Dotaro Usui
Abstract
Plastic bags are challenging objects for robot manipulation due to transparency and deformability. This paper proposes a learning approach for a robot to insert its hand into a container- or bag-shaped object based on visual and tactile sensing. The basic idea is to utilize probing action that allows to acquire rich information about the object even with a simple tactile sensor. The structure of the object is estimated by unsupervised learning with contact and reachability information. The result is transferred to visual recognition as self-supervised learning. Based on the unsupervised learning result, the robot can verify whether the hand truly reached the interior of the bag by additional probing actions. The proposed method was evaluated experimentally by a robot hand with a simple tactile sensor.
BibTeX
@inproceedings{iros2025_selfsupervisedco,
title = {Self-Supervised Complementary Learning between Vision and Tactility by Probing Action into an Open-Mouth Container},
author = {Daiki Takamori and Tomohiro Hayakawa and Yuichi Kobayashi and Kosuke Hara and Dotaro Usui},
booktitle = {IROS 2025},
year = {2025}
}