Benchmarking Long-Horizon Mobile Manipulation in Multi-Room Dynamic Environments

Abstract

Long-horizon reasoning and task execution are crucial for complex mobile manipulation tasks in household environments. Existing benchmarks and methods primarily focus on single-room or single-object mobile manipulation scenarios, limiting the scope of long-horizon planning and scene-level understanding. To address this gap, we introduce a novel benchmark for long-horizon mobile manipulation in multi-room household environments. Our task requires agents to follow a sequence of language instructions, each directing the movement of specific objects across receptacles and rooms. In this task, we investigate the role of long-term memory by constructing a hierarchical scene graph that captures the relationships between objects, furniture, and rooms. This scene graph-based memory is dynamically updated as the agent explores the environment, which effectively aligns the scene information with the targets and environmental context specified in the language instructions. Additionally, we benchmark the proposed task in dynamic environments where objects can be relocated during task execution, simulating real-world scenarios. Our results demonstrate that the scene graph-based memory significantly improves the agent’s performance in long-horizon mobile manipulation tasks. Moreover, dynamically updating the state of objects within the scene graph enables the agent to better adapt to dynamic conditions.