DGETP: Dynamic Graph Attention Network for Embodied Task Planning

Abstract

With the development of embodied intelligence, many studies have made progress by incorporating scene graphs and GNN into task planning. However, most methods still face challenges in fully capturing the sequential relationships between agent actions and the environment, making it difficult to handle dynamic changes and complexity inherent in embodied tasks. This paper proposes a Dynamic Graph Attention Network for Embodied Task Planning (DGETP) to process scene graph sequences and robot graphs for dynamic environment perception. In DGETP, we design a Hierarchical Dynamic Graph Attention network (H-DGAT) by employing both structural and temporal attention mechanisms to model the dynamic evolution feature of the scene. A Dual-branch Action-object Predictor (DAP) is proposed in DGETP through introducing sequences of previous actions and objects to efficiently aggregate historical information. DAP captures temporal dependencies between past and future actions through explicit sequence modeling, and reduces prediction complexity via a dual-branch architecture that separates action and object prediction while preserving their correlations through targeted feature fusion. Experiments show that DGETP improves task accuracy by over 30% in seen scenes and over 15% in unseen scenes compared to other baselines. In complex scenes, DGETP demonstrates strong generalization ability. Finally, the simulation environment indicates that DGETP achieves more goals than most of the advanced task planning method.