Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation
Martina Lippi, Petra Poklukar, Michael C. Welle, Anastasiia Varava, Hang Yin, Alessandro Marino, Danica Kragic
Abstract
We present a framework for visual action planning of complex manipulation tasks with high-dimensional state spaces such as manipulation of deformable objects. Planning is performed in a low-dimensional latent state space that embeds images. We define and implement a Latent Space Roadmap (LSR) which is a graph-based structure that globally captures the latent system dynamics. Our framework consists of two main components: a Visual Foresight Module (VFM) that generates a visual plan as a sequence of images, and an Action Proposal Network (APN) that predicts the actions between them. We show the effectiveness of the method on a simulated box stacking task as well as a T-shirt folding task performed with a real robot.
BibTeX
@inproceedings{iros2020_latentspaceroadm,
title = {Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation},
author = {Martina Lippi and Petra Poklukar and Michael C. Welle and Anastasiia Varava and Hang Yin and Alessandro Marino and Danica Kragic},
booktitle = {IROS 2020},
year = {2020}
}