Supportive Relationships-Aware Hierarchical Reinforcement Learning for Efficient Ex-Situ Object Rearrangement

Abstract

In ex-situ object rearrangement tasks within open environments, robots face significant challenges due to the increased cost of moving objects over large workspaces. To address this issue, we propose a hierarchical reinforcement learning-based approach that takes into account the supportive relationships and semantic correlations between objects. The robot groups and stacks objects with compatible supportive capabilities, moving them together to their target locations to optimize task execution. Specifically, we use a large language model to assess the supportive relationships and semantic correlations between objects. In the high-level decision-making process, objects are grouped based on their supportive capabilities, while the low-level process refines these groupings using a graph capsule convolutional network. Experimental results demonstrate that our approach not only reduces the number of movements required but also improves task efficiency and significantly decreases task completion time by approximately 50%, compared to methods that do not consider supportive relationships.