Dynamics-Aware Embeddings
William Whitney, Rajat Agarwal, Kyunghyun Cho, Abhinav Gupta
Abstract
In this paper we consider self-supervised representation learning to improve sample efficiency in reinforcement learning (RL). We propose a forward prediction objective for simultaneously learning embeddings of states and actions. These embeddings capture the structure of the environment's dynamics, enabling efficient policy learning. We demonstrate that our action embeddings alone improve the sample efficiency and peak performance of model-free RL on control from low-dimensional states. By combining state and action embeddings, we achieve efficient learning of high-quality policies on goal-conditioned continuous control from pixel observations in only 1-2 million environment steps.
BibTeX
@inproceedings{
Whitney2020Dynamics-Aware,
title={Dynamics-Aware Embeddings},
author={William Whitney and Rajat Agarwal and Kyunghyun Cho and Abhinav Gupta},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=BJgZGeHFPH}
}