SDA-LLM: Spatial DisAmbiguation via Multi-turn Vision-Language Dialogues for Robot Navigation

Abstract

When users give natural language instructions to service robots, positional information is often referenced relative to objects in the environment rather than absolute coordinates. However, humans naturally use relative references. For example, in“Go to the chair and pick up empty bottles”, where the positional reference is the chair, ambiguity arises when multiple similar objects co-exist in the environment or when the robot’s view is limited, resulting in multiple possible interpretations of the same command and affecting navigation decisions. To address this issue, we propose a two-level framework that integrates a large language model (LLM) and a vision-language model (VLM), allowing the robot to engage in multi-turn dialogues for spatial disambiguation. Our method first utilizes a VLM to map the semantic meanings of dialogues to a unique object ID in images and then further maps this object ID to a 3D depth map, enabling the robot to accurately determine its navigation target. To the best of our knowledge, this is the first work leveraging foundation models to address spatial ambiguity.