MVTrans: Multi-View Perception of Transparent Objects
Yi Ru Wang, Yuchi Zhao, Haoping Xu, Sagi Eppel, Alán Aspuru-Guzik, Florian Shkurti, Animesh Garg
Abstract
Transparent object perception is a crucial skill for applications such as robot manipulation in household and laboratory settings. Existing methods utilize RGB-D or stereo inputs to handle a subset of perception tasks including depth and pose estimation. However transparent object perception remains to be an open problem. In this paper, we forgo the unreliable depth map from RGB-D sensors and extend the stereo based method. Our proposed method, MVTrans, is an end-to-end multi-view architecture with multiple perception capabilities, including depth estimation, segmentation, and pose estimation. Additionally, we establish a novel procedural photo-realistic dataset generation pipeline and create a large-scale transparent object detection dataset, Syn-TODD, which is suitable for training networks with all three modalities, RGB-D, stereo and multi-view RGB. https://ac-rad.github.io/MVTrans/
BibTeX
@inproceedings{icra2023_mvtransmultiview,
title = {MVTrans: Multi-View Perception of Transparent Objects},
author = {Yi Ru Wang and Yuchi Zhao and Haoping Xu and Sagi Eppel and Alán Aspuru-Guzik and Florian Shkurti and Animesh Garg},
booktitle = {ICRA 2023},
year = {2023}
}