Imitation Learning with Approximated Behavior Cloning Loss

Abstract

Recent Imitation Learning (IL) techniques focus on adversarial imitation learning algorithms to learn from a fixed set of expert demonstrations. While these approaches are theoretically sound, they suffer from a number of problems such as poor sample efficiency, poor stability, and a host of issues that Generative Adversarial Networks (GANs) suffer from. In this paper we introduce a generalization of Behavior Cloning (BC) that is applicable in any IL setting. Our algorithm first approximates behavior cloning loss using a neural network and then uses that loss network to generate a loss signal which is minimized using standard supervised learning. We call the resulting algorithm family Approximated Behavior Cloning (ABC), introduce variants for each IL setting, and demonstrate an order of magnitude improvement in sample efficiency and increased stability in standard imitation learning environments.