Control Reallocation Using Deep Reinforcement Learning for Actuator Fault Recovery of an Autonomous Underwater Vehicle

Abstract

Actuator faults in dynamic systems pose significant challenges, particularly for robotic systems operating in hostile environments such as Autonomous Underwater Vehicles (AUVs), risking loss of stability and performance degradation. Fault Tolerant Control (FTC) strategies, including Control Reallocation (CR), have been developed to mitigate such risks. However, these strategies extensively depend on explicit fault diagnosis, which may present challenges regarding computational demands and efficiency, particularly when dealing with unknown faults. This paper presents a novel method that performs CR with Deep Reinforcement Learning (DRL) for actuator fault recovery without explicit fault diagnosis. The approach is implemented on a BlueROV2 underwater vehicle and demonstrates improved performance for fault recovery compared to a standard Proportional-Integral-Derivative (PID) controller and a variable gain PID controller, both in simulation and in real-world conditions. The DRL-based CR method demonstrates generalisability by successfully handling faults not encountered during training, highlighting its adaptability to unforeseen circumstances.