Interactive Object Detection by Mitigating Uncertainty of Robot Task Plans using Large Language Model
Kanata Suzuki, Akane Ushizaka, Kazuki Hori, Tetsuya Ogata
Abstract
Recently, many attempts have been made to integrate the foundation model with robotics. In most of those attempts, the model recognition results were treated as unique; however, the recognition results required for real robot tasks vary with the task goal. The recognition results of the foundation model are determined from the detection query; therefore, in the case of an ambiguous query, the query must be modified to match the purpose of the robot task. In this study, we propose an object recognition method that considers the task goal through application of an interactive task planning method using a large language model. The proposed method clarifies the purpose of the robot task by asking the user a question. Hence, uncertainty in the task plan due to ambiguous operation instructions is mitigated. During the task plan update arising from the dialog process, the object-detection results obtained from the query in the planning results are also updated to match the task goal. In our experiments, the proposed method’s effectiveness is verified quantitatively and qualitatively via object-detection tasks conducted on a custom-built verification dataset.
BibTeX
@inproceedings{iros2025_interactiveobjec,
title = {Interactive Object Detection by Mitigating Uncertainty of Robot Task Plans using Large Language Model},
author = {Kanata Suzuki and Akane Ushizaka and Kazuki Hori and Tetsuya Ogata},
booktitle = {IROS 2025},
year = {2025}
}