Online Planning for Multi-UAV Pursuit-Evasion in Unknown Environments Using Deep Reinforcement Learning

Abstract

Multi-UAV pursuit-evasion, where pursuers aim to capture evaders, poses a key challenge for UAV swarm intelligence. Multi-agent reinforcement learning (MARL) has demonstrated potential in modeling cooperative behaviors, but most RL-based approaches remain constrained to simplifed simulations with limited dynamics or fxed scenarios. Previous attempts to deploy RL policy to real-world pursuit-evasion are largely restricted to two-dimensional scenarios, such as ground vehicles or UAVs at fxed altitudes. In this paper, we propose a novel MARL-based algorithm that learns online planning for multi-UAV pursuitevasion in unknown environments (OPEN). OPEN introduces an evader prediction-enhanced network to tackle partial observability in cooperative policy learning. Additionally, OPEN proposes an adaptive environment generator within MARL training, enabling higher exploration effciency and better policy generalization across diverse scenarios. Simulations show our method signifcantly outperforms all baselines in challenging scenarios, generalizing to unseen scenarios with a 100% capture rate. Finally, after integrating calibrated dynamics models of UAVs into training, we derive a feasible policy via a two-stage reward refnement and deploy the policy on real quadrotors in a zero-shot manner. To our knowledge, this is the frst work to derive and deploy an RL-based policy using collective thrust and body rates control commands for multi-UAV pursuit-evasion in unknown environments. The open-source code and videos are available at https://sites.google. com/view/pursuit-evasion-rl.