Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes

Taylor W Killian; Samuel Daulton; George Konidaris; Finale Doshi-Velez

NeurIPS 2017oral138 citations

Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes

Taylor W Killian, Samuel Daulton, George Konidaris, Finale Doshi-Velez

Abstract

We introduce a new formulation of the Hidden Parameter Markov Decision Process (HiP-MDP), a framework for modeling families of related tasks using low-dimensional latent embeddings. Our new framework correctly models the joint uncertainty in the latent parameters and the state space. We also replace the original Gaussian Process-based model with a Bayesian Neural Network, enabling more scalable inference. Thus, we expand the scope of the HiP-MDP to applications with higher dimensions and more complex dynamics.

BibTeX

@inproceedings{NIPS2017_2227d753,
 author = {Killian, Taylor W and Daulton, Samuel and Konidaris, George and Doshi-Velez, Finale},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {I. Guyon and U. Von Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
 pages = {},
 publisher = {Curran Associates, Inc.},
 title = {Robust and Efficient Transfer Learning with Hidden Parameter Markov Decision Processes},
 url = {https://proceedings.neurips.cc/paper_files/paper/2017/file/2227d753dc18505031869d44673728e2-Paper.pdf},
 volume = {30},
 year = {2017}
}