NeurIPS 2021poster15 citations
Imitation with Neural Density Models
Kuno Kim, Akshat Jindal, Yang Song, Jiaming Song, Yanan Sui, Stefano Ermon
Abstract
We propose a new framework for Imitation Learning (IL) via density estimation of the expert's occupancy measure followed by Maximum Occupancy Entropy Reinforcement Learning (RL) using the density as a reward. Our approach maximizes a non-adversarial model-free RL objective that provably lower bounds reverse Kullback–Leibler divergence between occupancy measures of the expert and imitator. We present a practical IL algorithm, Neural Density Imitation (NDI), which obtains state-of-the-art demonstration efficiency on benchmark control tasks.
Imitation LearningReinforcement LearningDensity EstimationDensity ModelMaximum Entropy RLMujoco
BibTeX
@inproceedings{
kim2021imitation,
title={Imitation with Neural Density Models},
author={Kuno Kim and Akshat Jindal and Yang Song and Jiaming Song and Yanan Sui and Stefano Ermon},
booktitle={Advances in Neural Information Processing Systems},
editor={A. Beygelzimer and Y. Dauphin and P. Liang and J. Wortman Vaughan},
year={2021},
url={https://openreview.net/forum?id=cMv0gvg88a}
}