An Attention-aware Deep Reinforcement Learning Framework for UAV-UGV Collaborative Route Planning

Abstract

Unmanned aerial vehicles (UAVs) possess the capability to survey vast areas, yet their operational range is limited by their battery capacity. Deploying mobile recharging stations via unmanned ground vehicles (UGVs) can significantly enhance the endurance and effectiveness of UAVs. However, optimizing the routes for both UAVs and UGVs, referred to as the UAV-UGV cooperative routing problem, requires a sophisticated planning framework to determine the vehicles’ routes and their recharging points. To address this, in this paper, we utilize a deep reinforcement learning (DRL) based framework equipped with multi-head attention layers. The framework is designed to sequentially select actions to construct routes for the UAV and UGV and to establish their rendezvous points for recharging. We evaluate our framework across various problem instance sizes and distributions, comparing it against recent heuristic-based methods and an existing learning-based method as baselines. Our proposed algorithm surpasses these baselines in terms of solution quality and runtime efficiency in the test scenarios, thus proving its effectiveness. Additionally, we investigate the application of our DRL policy in online mission planning to accommodate dynamic changes within the mission scenario.