Regularizing Trajectory Optimization with Denoising Autoencoders
Rinu Boney, Norman Di Palo, Mathias Berglund, Alexander Ilin, Juho Kannala, Antti Rasmus, Harri Valpola
Abstract
Trajectory optimization using a learned model of the environment is one of the core elements of model-based reinforcement learning. This procedure often suffers from exploiting inaccuracies of the learned model. We propose to regularize trajectory optimization by means of a denoising autoencoder that is trained on the same trajectories as the model of the environment. We show that the proposed regularization leads to improved planning with both gradient-based and gradient-free optimizers. We also demonstrate that using regularized trajectory optimization leads to rapid initial learning in a set of popular motor control tasks, which suggests that the proposed approach can be a useful tool for improving sample efficiency.
BibTeX
@inproceedings{NEURIPS2019_21fe5b8b,
author = {Boney, Rinu and Di Palo, Norman and Berglund, Mathias and Ilin, Alexander and Kannala, Juho and Rasmus, Antti and Valpola, Harri},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Regularizing Trajectory Optimization with Denoising Autoencoders},
url = {https://proceedings.neurips.cc/paper_files/paper/2019/file/21fe5b8ba755eeaece7a450849876228-Paper.pdf},
volume = {32},
year = {2019}
}