Neural networks for incremental dimensionality reduced reinforcement learning

Abstract

State-of-the-art personal robots must perform complex manipulation tasks to be viable in assistive scenarios. However, many of these robots, like the PR2, use manipulators with high degrees-of-freedom. The complexity of these robots lead to large dimensional state spaces, which are difficult to fully explore. Our previous work introduced the IDRRL algorithm, which compresses the learning space by transforming a high-dimensional learning space onto a lower-dimensional manifold while preserving expressivity. In this work we formally prove that IDRRL maintains PAC-MDP guarantees. We then improve upon our previous formulation of IDRRL by introducing cascading autoencoders (CAE) for dimensionality reduction, producing the new algorithm IDRRL-CAE. We demonstrate the improvement of this extension over our previous formulation, IDRRL-PCA, in the Mountain Car and Swimmers domains.