Look and Listen: A Multi-Sensory Pouring Network and Dataset for Granular Media from Human Demonstrations
Alexis Burns, Siyuan Xiang, Daewon Lee, Larry Jackel, Shuran Song, Volkan Isler
Abstract
Humans have the ability to pour various media, both liquid and granular, to desired ends in various containers. We do this by using multiple senses simultaneously in a constant feedback loop to complete a pouring task. Combining multiple sensing modalities, similar to humans, could aid in robotic pouring control outside of a structured or industrial setting. We present a multi-sensory pouring dataset consisting of human pouring demonstrations of various granular media, coupled with two multi-sensory networks that estimate pouring rate and pouring average height. For both pouring metrics, a combined input of audio and visual data provides a lower median error than either the audio network or visual network. The multi-sensory network achieves a median error of 6.4 mm for average height estimation and 0.06 N/s for pouring rate estimation.
BibTeX
@inproceedings{icra2022_lookandlistenamu,
title = {Look and Listen: A Multi-Sensory Pouring Network and Dataset for Granular Media from Human Demonstrations},
author = {Alexis Burns and Siyuan Xiang and Daewon Lee and Larry Jackel and Shuran Song and Volkan Isler},
booktitle = {ICRA 2022},
year = {2022}
}