Multifunctional Proximity-Aware Skin With Multizone ToF Sensing for AR-Assisted Human-Robot Interactive Telesurgery
Zheng Xu, Zecai Lin, Sihong Zuo, Shaoping Huang, Junhao Zhao, Yun Zou, Jiafan Chen, Anzhu Gao
Abstract
Augmented Reality (AR)-assisted telesurgery has proven its effectiveness in enhancing situational awareness for clinician-side teleoperators. However, for patient-side surgical assistants, existing systems lack intuitive and safe interaction strategies to mitigate preoperative infection risks and intraoperative collisions. To address these challenges, this paper introduces a multifunctional proximity-aware skin that integrates multizone Time-of-Flight (ToF) sensing for interaction during both preoperative and intraoperative phases of AR-assisted telesurgery. On the patient side, a monocular camera providing AR visualization is mounted at the end of a robotic arm equipped with the skin. The skin uses a time-division multiplexing strategy to eliminate crosstalk and ensure reliable proximity awareness. During preoperative preparation, the skin-based contactless teleoperation allows surgical assistants to adjust the AR robotic arm for a high-quality field of view, while the contactless gesture recognition enables mode switching between rotation, translation, and locking. In the intraoperative phase, the skin facilitates obstacle avoidance to ensure patient-side safe assistance. Accuracy evaluations demonstrate effective crosstalk suppression under the proposed time-division multiplexing strategy. Functional experiments, laparoscopic phantom and tissue tests, as well as user studies validate the system's effectiveness to improve patient-side assistance.
BibTeX
@inproceedings{ral2026_multifunctionalp,
title = {Multifunctional Proximity-Aware Skin With Multizone ToF Sensing for AR-Assisted Human-Robot Interactive Telesurgery},
author = {Zheng Xu and Zecai Lin and Sihong Zuo and Shaoping Huang and Junhao Zhao and Yun Zou and Jiafan Chen and Anzhu Gao},
booktitle = {RA-L 2026},
year = {2026}
}