Curious iLQR: Resolving Uncertainty in Model-based RL
Sarah Bechtle, Yixin Lin, Akshara Rai, Ludovic Righetti, Franziska Meier
Abstract
Curiosity as a means to explore during reinforcement learning problems has recently become very popular. However, very little progress has been made in utilizing curiosity for learning control. In this work, we propose a model-based reinforcement learning (MBRL) framework that combines Bayesian modeling of the system dynamics with curious iLQR , an iterative LQR approach that considers model uncertainty. During trajectory optimization the curious iLQR attempts to minimize both the task-dependent cost and the uncertainty in the dynamics model. We demonstrate the approach on reaching tasks with 7-DoF manipulators in simulation and on a real robot. Our experiments show that MBRL with curious iLQR reaches desired end-effector targets more reliably and with less system rollouts when learning a new task from scratch, and that the learned model generalizes better to new reaching tasks.
BibTeX
@inproceedings{corl2019_curiousilqrresol,
title = {Curious iLQR: Resolving Uncertainty in Model-based RL},
author = {Sarah Bechtle and Yixin Lin and Akshara Rai and Ludovic Righetti and Franziska Meier},
booktitle = {CoRL 2019},
year = {2019}
}