Stability and Transparency in Mixed Reality Bilateral Human Teleoperation

Abstract

Recent work introduced human teleoperation (HT), where the remote robot typically used in conventional bilateral teleoperation is replaced by a novice person wearing a mixed reality headset and tracking the motion of a virtual tool controlled by an expert. HT has advantages in cost, complexity, and patient acceptance for telemedicine in remote or low-resource communities. However, the stability, transparency, and performance of bilateral HT are unexplored. In this paper, we therefore develop a mathematical model of the HT system using test data. We then analyze various control architectures with this model and implement them with the HT system, testing volunteer operators and a virtual fixture-based simulated patient to find the achievable performance, investigate stability, and determine the most promising teleoperation scheme in the presence of time delays. We show that instability in HT, while not destructive or dangerous, makes the system unusable. However, stable and transparent teleoperation are possible with small time delays (<200 ms) through 3-channel teleoperation, or with large delays through model-mediated teleoperation with local pose and force feedback for the novice.