A Composable Specification Language for Reinforcement Learning Tasks
Kishor Jothimurugan, Rajeev Alur, Osbert Bastani
Abstract
Reinforcement learning is a promising approach for learning control policies for robot tasks. However, specifying complex tasks (e.g., with multiple objectives and safety constraints) can be challenging, since the user must design a reward function that encodes the entire task. Furthermore, the user often needs to manually shape the reward to ensure convergence of the learning algorithm. We propose a language for specifying complex control tasks, along with an algorithm that compiles specifications in our language into a reward function and automatically performs reward shaping. We implement our approach in a tool called SPECTRL, and show that it outperforms several state-of-the-art baselines.
BibTeX
@inproceedings{NEURIPS2019_f5aa4bd0,
author = {Jothimurugan, Kishor and Alur, Rajeev and Bastani, Osbert},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {A Composable Specification Language for Reinforcement Learning Tasks},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/f5aa4bd09c07d8b2f65bad6c7cd3358f-Paper.pdf},
volume = {32},
year = {2019}
}