Leveraging Language for Accelerated Learning of Tool Manipulation
Allen Z. Ren, Bharat Govil, Tsung-Yen Yang, Karthik R Narasimhan, Anirudha Majumdar
Abstract
Robust and generalized tool manipulation requires an understanding of the properties and affordances of different tools. We investigate whether linguistic information about a tool (e.g., its geometry, common uses) can help control policies adapt faster to new tools for a given task. We obtain diverse descriptions of various tools in natural language and use pre-trained language models to generate their feature representations. We then perform language-conditioned meta-learning to learn policies that can efficiently adapt to new tools given their corresponding text descriptions. Our results demonstrate that combining linguistic information and meta-learning significantly accelerates tool learning in several manipulation tasks including pushing, lifting, sweeping, and hammering.
BibTeX
@inproceedings{
ren2022leveraging,
title={Leveraging Language for Accelerated Learning of Tool Manipulation},
author={Allen Z. Ren and Bharat Govil and Tsung-Yen Yang and Karthik R Narasimhan and Anirudha Majumdar},
booktitle={6th Annual Conference on Robot Learning},
year={2022},
url={https://openreview.net/forum?id=nPw7jaGBrCG}
}