Model-based Domain Randomization of Dynamics System with Deep Bayesian Locally Linear Embedding

Abstract

Domain randomization (DR) is a powerful tool to make a policy robust to the uncertainty of dynamics caused by unobservable environmental parameters. Conventional DR has adopted model-free reinforcement learning as a policy optimizer. However, the model-free methods in DR demand high time-complexity due to the randomization process where the environment is extremely changed. In this paper, we introduce model-based dynamics and policy learning for efficient DR. A Bayesian model of locally linear embedding is designed to fit the stochastic dynamics in DR. By virtue of locally linear dynamics, model-based optimal control is substituted for the policy optimization. Unlike previous works, our proposed Bayesian model with a MNIW prior allows the locally linear embedding to capture the dynamics in DR as a stochastic model. We show that a training method that combines variational and adversarial approaches is adequate for Bayesian embedding. Finally, a model-based controller is designed on our Bayesian locally linear embedding, and it shows better performance in DR environments compared with the non-Bayesian model of locally linear embedding.