A Monocular Vision-based Robotic Arm Teleoperation Method for Human Arm Configuration Imitation

Abstract

Imitation-based teleoperation enables intuitive robot control in hazardous or hard-to-reach environments. Existing methods, however, lack an effective and quickly-deployable system that uses simple visual sensors to achieve end-effector control and human-like arm joint configuration imitation across various robotic arm structures. This paper therefore presents a teleoperation system that utilizes a single RGB camera and advanced computer vision techniques to capture human motion, coupled with a kinematic mapping method to transfer movements from human to robotic arms. The system generates robot motion that ensures both end-effector tracking and human-like joint configuration imitation, adaptable to diverse structures, including those with multiple offset links. Experiments demonstrate that the system produces robot arm poses more closely aligned with human configurations compared to traditional methods that overlook human pose. The performance of the end-effector tracking control and human arm shape imitation is evaluated, with no noticeable error observed when the robot completes its motion and a maximum position error of 17.03% and a maximum orientation error of 0.0925 rad are observed during motion, which are likely attributed to delays cased by filters and communications. Additionally, the system’s ability to actively avoid obstacles via arm configuration imitation in specific scenarios is confirmed. Supplementary video is available.