Lucas Manuelli — accepted AI-conference papers

2023

Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangement

Anthony Simeonov, Ankit Goyal, Lucas Manuelli, Yen-Chen Lin, Alina Sarmiento, Alberto Rodriguez Garcia +2

CoRL 2023poster

We propose a system for rearranging objects in a scene to achieve a desired object-scene placing relationship, such as a book inserted in an open slot of a bookshelf. The pipeline generalizes to novel geometries, poses, and layouts of both scenes and objects, and is trained from demonstrations to op…

Cited by 46 Source ↗code ↗Scholar ↗

2022

MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare

Yann Labbé, Lucas Manuelli, Arsalan Mousavian, Stephen Tyree, Stan Birchfield, Jonathan Tremblay +4

CoRL 2022poster

We introduce MegaPose, a method to estimate the 6D pose of novel objects, that is, objects unseen during training. At inference time, the method only assumes knowledge of (i) a region of interest displaying the object in the image and (ii) a CAD model of the observed object. The contributions of thi…

Cited by 157 Source ↗code ↗Scholar ↗

2022

Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation

Mohit Shridhar, Lucas Manuelli, Dieter Fox

CoRL 2022poster

Transformers have revolutionized vision and natural language processing with their ability to scale with large datasets. But in robotic manipulation, data is both limited and expensive. Can manipulation still benefit from Transformers with the right problem formulation? We investigate this question…

Cited by 530 Source ↗code ↗Scholar ↗

2021

CLIPort: What and Where Pathways for Robotic Manipulation

Mohit Shridhar, Lucas Manuelli, Dieter Fox

CoRL 2021poster

How can we imbue robots with the ability to manipulate objects precisely but also to reason about them in terms of abstract concepts? Recent works in manipulation have shown that end-to-end networks can learn dexterous skills that require precise spatial reasoning, but these methods often fail to ge…

Cited by 728 Source ↗code ↗Scholar ↗

2020

Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning

Lucas Manuelli, Yunzhu Li, Pete Florence, Russ Tedrake

CoRL 2020

Predictive models have been at the core of many robotic systems, from quadrotors to walking robots. However, it has been challenging to develop and apply such models to practical robotic manipulation due to high-dimensional sensory observations such as images. Previous approaches to learning models

Cited by 0 Source ↗PDF ↗Scholar ↗

2020

Self-Supervised Correspondence in Visuomotor Policy Learning

Peter R. Florence, Lucas Manuelli, Russ Tedrake

RA-L 2020

In this letter, we explore using self-supervised correspondence for improving the generalization performance and sample efficiency of visuomotor policy learning. Prior work has primarily used approaches such as autoencoding, pose-based losses, and end-to-end policy optimization in order to train the

Cited by 186 Source ↗Scholar ↗

2018

Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation

Peter R. Florence, Lucas Manuelli, Russ Tedrake

CoRL 2018

What is the right object representation for manipulation? We would like robots to visually perceive scenes and learn an understanding of the objects in them that (i) is task-agnostic and can be used as a building block for a variety of manipulation tasks, (ii) is generally applicable to both rigid a

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗

2018

Label Fusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes

Pat Marion, Peter R. Florence, Lucas Manuelli, Russ Tedrake

ICRA 2018poster

Deep neural network (DNN) architectures have been shown to outperform traditional pipelines for object segmentation and pose estimation using RGBD data, but the performance of these DNN pipelines is directly tied to how representative the training data is of the true data. Hence a key requirement fo…

Cited by 138 Source ↗Scholar ↗

2016

Localizing external contact using proprioceptive sensors: The Contact Particle Filter

Lucas Manuelli, Russ Tedrake

IROS 2016poster

In order for robots to interact safely and intelligently with their environment they must be able to reliably estimate and localize external contacts. This paper introduces CPF, the Contact Particle Filter, which is a general algorithm for detecting and localizing external contacts on rigid body rob…

Cited by 120 Source ↗Scholar ↗