Zero-Shot Policy Transfer with Disentangled Task Representation of Meta-Reinforcement Learning
Zheng Wu, Yichen Xie, Wenzhao Lian, Changhao Wang, Yanjiang Guo, Jianyu Chen, Stefan Schaal, Masayoshi Tomizuka
Abstract
Humans are capable of abstracting various tasks as different combinations of multiple attributes. This perspective of compositionality is vital for human rapid learning and adaption since previous experiences from related tasks can be combined to generalize across novel compositional settings. In this work, we aim to achieve zero-shot policy generalization of Reinforcement Learning (RL) agents by leveraging the task compositionality. Our proposed method is a meta-RL algorithm with disentangled task representation, explicitly encoding different aspects of the tasks. Policy generalization is then performed by inferring unseen compositional task representations via the obtained disentanglement without extra exploration. The evaluation is conducted on three simulated tasks and a challenging real-world robotic insertion task. Experimental results demonstrate that our proposed method achieves policy generalization to unseen compositional tasks in a zero-shot manner.
BibTeX
@inproceedings{icra2023_zeroshotpolicytr,
title = {Zero-Shot Policy Transfer with Disentangled Task Representation of Meta-Reinforcement Learning},
author = {Zheng Wu and Yichen Xie and Wenzhao Lian and Changhao Wang and Yanjiang Guo and Jianyu Chen and Stefan Schaal and Masayoshi Tomizuka},
booktitle = {ICRA 2023},
year = {2023}
}