Multi-Drone-Truck Collaborative Delivery with En Route Operations: A Hierarchical MARL-Based Approach

Abstract

The multi-drone-truck collaborative delivery, where unmanned trucks serve as mobile supply stations for drones, effectively combines the strengths of both vehicles and presents wide application prospects. But the majority of existing literature restricts drone launch and retrieve operations (LARO) to stationary trucks, and potential drone route collisions are mostly ignored. This leads to inability to fully exploit the capability of drones. We address these gaps and introduce a new variant of multi-drone-truck collaborative delivery. However, the scheduling for drones and truck faces high-dimensional solution space and complex constraints, making it almost impossible for centralized solving. To this end, we develop a hierarchical solution framework that decomposes the complete problem into two levels of subproblem. The upper solver centrally allocates tasks and schedules when drones to launch, while the lower solver, based on multi-agent reinforcement learning (MARL), plans paths for each drone agent in a decentralized but cooperative manner. In addition, we validate the effectiveness of our method by benchmarking it against three state-of-the-art approaches, demonstrating its superiority in terms of both efficiency and collision avoidance.