An Intention-Guided Reinforcement Learning Approach With Dirichlet Energy Constraint for Heterogeneous Multi-Robot Cooperation

Abstract

Multi-robot systems have demonstrated significant potential in accomplishing complex tasks, such as cooperative pursuit, search-and-rescue operations. The emergence of heterogeneous robots with diverse capabilities and characteristics shows superior adaptability compared with homogeneous teams. However, in practical applications, global information is typically inaccessible, and composite teams must contend with partial observability and coordination difficulties. To address the issue in heterogeneous multi-robot systems, we propose a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</i>ntention-<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</i>uided reinforcement learning approach with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</i>irichlet <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</i>nergy constraint (IGDE). Specifically, an intention-guided module is designed to derive long-horizon strategies based solely on local observations, enabling foresighted decision-making. In addition, a Dirichlet energy constraint is incorporated into the communication process to enhance the diversity of environmental cognition among different classes of robots. Heterogeneous robots perform class-aware actions driven by distinct cognitive representations, thereby enhancing cooperative efficiency. Notably, our approach alleviates the need of prior knowledge and heterogeneity modeling. Extensive comparative experiments and ablation studies verify the effectiveness of the proposed framework. Additionally, real-world deployment is conducted to demonstrate the practicality.