A Safe and Convenient Feeding Assistive Robotic Based on Multi-modal Interaction Method
Jiahui Ding, Donghui Zhao, Baixue Liu, Junyou Yang, Shuoyu Wang, Houde Liu, Toshio Fukuda
Abstract
For individuals with limited mobility who are bedridden for extended periods, providing comfortable assisted feeding services is one of the most significant actions to enhance their quality of life. Despite the development of various feeding assistive robots, there remain limitations in terms of interaction convenience and safety, which restrict the overall feeding experience for users. To address these challenges, this study first establishes a feeding assistive robot system that integrates multimodal interaction methods. Furthermore, we propose an interactive feeding method that combines both safety and comfort. This method utilizes visual recognition to detect the user’s active meal intent, food selection preferences, and chewing status. Additionally, based on a Large Language Model (LLM), a monitoring thread is designed to conduct voice interactions regarding the user’s ambiguous intentions, temporary changes in intent, emergency situations, and risky behaviors throughout the feeding process. Comprehensive experimental results demonstrate that the proposed multimodal interaction method, which aligns with the natural eating patterns, incorporates both language and visual interactions, the two most convenient forms for users. It also matches force sensing and pose control techniques during the feeding stages, thereby enhancing the flexibility and safety of the assisted feeding system.
BibTeX
@inproceedings{iros2025_asafeandconvenie,
title = {A Safe and Convenient Feeding Assistive Robotic Based on Multi-modal Interaction Method},
author = {Jiahui Ding and Donghui Zhao and Baixue Liu and Junyou Yang and Shuoyu Wang and Houde Liu and Toshio Fukuda},
booktitle = {IROS 2025},
year = {2025}
}