A Deep Reinforcement Learning Method for Collision Avoidance with Dense Speed-Constrained Multi-UAV

Abstract

This letter introduces a novel deep reinforcement learning (DRL) method for collision avoidance problem of fixed-wing unmanned aerial vehicles (UAVs). First, with considering the characteristics of collision avoidance problem, a collision prediction method is proposed to identify the neighboring UAVs with a significant threat. A convolutional neural network model is devised to extract the dynamic environment features. Second, a trajectory tracking macro action is incorporated into the action space of the proposed DRL-based algorithm. Guided by the reward function that considers to reward for closing to the preset flight paths, UAVs could return to the preset flight path after completing the collision avoidance. The proposed method is trained in simulation scenarios, with model updates implemented using a soft actor-critic (SAC) algorithm. Validation experiments are conducted in several complex multi-UAV flight environments. The results demonstrate the superiority of our method over other advanced methods.