Nonparametric Bayesian reward segmentation for skill discovery using inverse reinforcement learning
Pravesh Ranchod, Benjamin Rosman, George Konidaris
Abstract
We present a method for segmenting a set of unstructured demonstration trajectories to discover reusable skills using inverse reinforcement learning (IRL). Each skill is characterised by a latent reward function which the demonstrator is assumed to be optimizing. The skill boundaries and the number of skills making up each demonstration are unknown. We use a Bayesian nonparametric approach to propose skill segmentations and maximum entropy inverse reinforcement learning to infer reward functions from the segments. This method produces a set of Markov Decision Processes (MDPs) that best describe the input trajectories. We evaluate this approach in a car driving domain and a simulated quadcopter obstacle course, showing that it is able to recover demonstrated skills more effectively than existing methods.
BibTeX
@inproceedings{iros2015_nonparametricbay,
title = {Nonparametric Bayesian reward segmentation for skill discovery using inverse reinforcement learning},
author = {Pravesh Ranchod and Benjamin Rosman and George Konidaris},
booktitle = {IROS 2015},
year = {2015}
}