Apprenticeship learning based on inconsistent demonstrations

Abstract

Apprenticeship learning based on inconsistent demonstrations is presented in this paper. We address a problem where given demonstrations are not directly applicable to reward function estimation due to the non-stationarity of an environment or the difference between the dynamics of a robot and a demonstrator. A basic idea of the proposed method is to use a subset of the trajectories sampled from the baseline policy as training data for inverse reinforcement learning. All consistent sample trajectories and inconsistent demonstrations are abstracted by an affine transformation invariant feature. Using the feature, the importance of each sample trajectory is estimated. Rating the sample trajectories based on importance, the training data for inverse reinforcement learning are identified. The validity of our approach is verified through simulation in two scenarios: inconsistency caused by variation of an environment and performance of a robot.