Viiat-Hand: A Reach-and-Grasp Restoration System Integrating Voice Interaction, Computer Vision, Auditory and Tactile Feedback for Non-Sighted Amputees
Chunhao Peng, Dapeng Yang, Deyu Zhao, Ming Cheng, Jinghui Dai, Li Jiang
Abstract
For non-sighted and visually impaired (BVI) amputees, the combined loss of vision and grasping abilities turns the seemingly simple task of reaching and grasping into a significant challenge. This letter introduces a novel multi-sensory prosthesis system designed for BVI amputees to assist in perception, navigation, and grasping tasks. The system integrates voice interaction, environmental perception, grasp guidance, collaborative control, and auditory/tactile feedback. Specifically, it processes user commands, provides environmental data via auditory/tactile channels, and manages collaborative control of grasp gestures and wrist angles for stable object handling. The prototype, viiat-hand, was experimentally tested with eight non-disabled and four non-sighted subjects performing reach-and-grasp tasks, showing that users could accurately reach (average time:15.24 s) and securely grasp objects (average time:17.23 s) in an indoor setting. The system also proved to be user-friendly, requiring minimal training for users to become adept.
BibTeX
@inproceedings{ral2024_viiathandareacha,
title = {Viiat-Hand: A Reach-and-Grasp Restoration System Integrating Voice Interaction, Computer Vision, Auditory and Tactile Feedback for Non-Sighted Amputees},
author = {Chunhao Peng and Dapeng Yang and Deyu Zhao and Ming Cheng and Jinghui Dai and Li Jiang},
booktitle = {RA-L 2024},
year = {2024}
}