From Demand to Grounded Plan: Task Customization and Planning for Service Robots With Deep Learning and LLMs

Abstract

In the field of task planning for service robots, large language model (LLM)-based approaches have shown increasing potential but still struggle with responding to complex user demands and grounded task planning. In this letter, we propose a user-demand-oriented adaptive grounded task planning system that integrates three modules: User-Centric Adaptive Task Customization (UATC), Task-Driven Scene Graph Pruning (TSGP), and Grounded Task Planning (GTP). UATC utilizes semantic demand-task contrastive learning to develop a novel demand parsing model for user natural language, generating personalized tasks that align with user needs and environmental constraints. TSGP employs LLM-based scene graph pruning and recursive traversal strategies to extract task-relevant environmental information. GTP generates an initial task plan using LLMs through action selection and effect judgment, followed by a three-stage refinement incorporating environmental constraints to ensure precise grounding of actions. Experimental results demonstrate that our approach can customize executable tasks in response to diverse user demands, and achieves strong planning performance with an executability rate of 91.39% and a task success rate of 78.33% in VirtualHome, outperforming existing state-of-the-art baselines. Moreover, a real-world deployment on the TIAGo robot further validates the system's applicability in physical environments.