f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
Xin Zhang, Yanhua Li, Ziming Zhang, Zhi-Li Zhang
Abstract
Imitation learning (IL) aims to learn a policy from expert demonstrations that minimizes the discrepancy between the learner and expert behaviors. Various imitation learning algorithms have been proposed with different pre-determined divergences to quantify the discrepancy. This naturally gives rise to the following question: Given a set of expert demonstrations, which divergence can recover the expert policy more accurately with higher data efficiency? In this work, we propose f-GAIL – a new generative adversarial imitation learning model – that automatically learns a discrepancy measure from the f-divergence family as well as a policy capable of producing expert-like behaviors. Compared with IL baselines with various predefined divergence measures, f-GAIL learns better policies with higher data efficiency in six physics-based control tasks.
BibTeX
@inproceedings{NEURIPS2020_967990de,
author = {Zhang, Xin and Li, Yanhua and Zhang, Ziming and Zhang, Zhi-Li},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {12805--12815},
publisher = {Curran Associates, Inc.},
title = {f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning},
url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/967990de5b3eac7b87d49a13c6834978-Paper.pdf},
volume = {33},
year = {2020}
}