Reinforcement Learning for Multi-Agent Path Finding in Large-Scale Warehouses via Distributed Policy Evolution

Abstract

Efficient multi-agent path finding (MAPF) is essential for large-scale warehousing and logistics systems. Despite the potential of reinforcement learning (RL) methods, current approaches struggle with challenges such as inefficient exploration, poor generalization and inadequate deadlock resolution. To address these issues, we propose a novel evolutionary reinforcement learning (ERL) framework to address the MAPF problem in large-scale warehouse environments. Specifically, the framework leverages distributed policy evolution methods to provide diverse experiences, thereby improving policy training efficiency and policy performance. We further integrate curriculum learning into this framework to improve the generality of the policy and make it scalable to larger environments. Additionally, we introduce a deadlock-breaking mechanism based on expert experience, helping to mitigate deadlock issues in large-scale and high-density scenarios. Experiments show that our method outperforms existing methods across various environments, particularly excelling in complex scenarios with over 1,000 agents.