Robust Deep Reinforcement Learning for Underwater Navigation with Unknown Disturbances

Abstract

We study an underwater navigation problem, where an Under-water Autonomous Vehicle must reach a target position in the presence of a disturbance that may be unknown. In order to deal with this problem, we make use of Deep Reinforcement Learning tools, and more concretely, we make use of robust control ideas, which allow training an agent in the presence of uncertainty. We propose a robust Proximal Policy Optimization agent and train it using simulations of an underwater medium: this agent shows an excellent performance when facing unknown disturbances, being able to approach the performance of the optimal agent which had an exact knowledge of the underwater disturbance.