Teleoperated Teaching of Task and Impedance (TTTI): Multi-Modal Interface Extending Haptic Device for Robotic Skill Transfer

Abstract

In this paper, we propose a concept of Teleoperated Teaching of Task and Impedance (TTTI) with a novel multi-modal interface that enables online teleoperated teaching of combined low-level impedance-regulation skills and high-level task decision-making skills using a single hand-held haptic device. To this end, we interactively switch the functionality of the haptic device for two modes of operation. To teach impedance-regulation low-level skills, we developed a novel stiffness command interface where the human operator uses the haptic device to manipulate the stiffness ellipsoid of the remote robotic arm endpoint in 3D space. For teaching high-level skills of how and when to employ low-level actions, we developed a GUI that enables a haptic device to remotely modify Behaviour Trees used to encode the robot’s task decision-making process. The interface connects both teaching modes, where a newly demonstrated low-level skill appears in the Behaviour Tree at an operator-specified index. To demonstrate the main features of the proposed interface, we performed several proof-of-concept experiments on a teleoperation setup operating a remote shelf-stocker robot in a simulated supermarket environment. We examined the task of placing a product on a shelf that consists of several sub-tasks, where each involves different stiffness strategies, while the Behaviour Tree has to encode the task sequencing and decision-making process.