Combining self-supervised learning and imitation for vision-based rope manipulation
Ashvin Nair, Dian Chen, Pulkit Agrawal, Phillip Isola, Pieter Abbeel, Jitendra Malik, Sergey Levine
Abstract
Manipulation of deformable objects, such as ropes and cloth, is an important but challenging problem in robotics. We present a learning-based system where a robot takes as input a sequence of images of a human manipulating a rope from an initial to goal configuration, and outputs a sequence of actions that can reproduce the human demonstration, using only monocular images as input. To perform this task, the robot learns a pixel-level inverse dynamics model of rope manipulation directly from images in a self-supervised manner, using about 60K interactions with the rope collected autonomously by the robot. The human demonstration provides a high-level plan of what to do and the low-level inverse model is used to execute the plan. We show that by combining the high and low-level plans, the robot can successfully manipulate a rope into a variety of target shapes using only a sequence of human-provided images for direction.
BibTeX
@inproceedings{icra2017_combiningselfsup,
title = {Combining self-supervised learning and imitation for vision-based rope manipulation},
author = {Ashvin Nair and Dian Chen and Pulkit Agrawal and Phillip Isola and Pieter Abbeel and Jitendra Malik and Sergey Levine},
booktitle = {ICRA 2017},
year = {2017}
}