Wearing a Robotic Hand to Feel 3D Force Feedback: Analysis and Virtual Reality Application of the Hand-in-Hand System

Abstract

Virtual Reality (VR) robot embodiment is a popular method for teleoperation and generating data to train AI control systems. A potential flaw with this approach is an incongruence in human-robot haptics. Without tactile feedback, teleoperators depend on visual cues, leading to suboptimal performance (slow movement, faulty grasps, excessive contact force). This problem is exacerbated during robot hand teleoperation. Worse, many haptic hand wearables are cable-driven and only produce unidirectional resistive force, thus failing to cover the gamut of 3D finger interactions. In this paper, a critical gap in teleoperated human-robot hand haptics is addressed by transforming an inexpensive (~500 USD) 3D-printed robotic hand into a wearable exoskeleton—the "Hand-in-Hand" (HiH) system—which provides fingertip 3D force feedback. Methods for force control, null-space optimization, and human finger pose estimation are presented and experimentally validated. Each finger of the device can produce a maximum of 1N of force feedback in any 3D direction. HiH finger tracking is compared to an industry-grade device (MANUS Metagloves, 5000 USD) and realizes an average inferred joint position Normalized Root Mean Square Error of 33.46%. Lastly, the HiH is demonstrated within a VR robot embodiment experiment with force feedback. Operator hand-manipulation performance improved when using force feedback, emphasizing the HiH’s potential for teleoperated robot control and data collection.