Hierarchical Deep Reinforcement Learning for Computation Offloading in Autonomous Multi-Robot Systems

Abstract

To ensure system responsiveness, some compute-intensive tasks are usually offloaded to cloud or edge computing devices. In environments where connection to external computing facilities is unavailable, computation offloading among members within an autonomous multi-robot system (AMRS) becomes a solution. The challenge lies in how to maximize the use of other members' idle resources without disrupting their local computation tasks. Therefore, this study proposes HRL-AMRS, a hierarchical deep reinforcement learning framework designed to distribute computational loads and reduce the processing time of computational tasks within an AMRS. In this framework, the high-level must consider the impact of data loading scales determined by low-level under varying computational device states on the actual processing times. In addition, the low-level employs Long Short-Term Memory (LSTM) networks to enhance the understanding of time-series states of computing devices. Experimental results show that, across various task sizes and numbers of robots, the framework reduces processing times by an average of 4.32% compared to baseline methods.