Efficient Visual Perception of Human-Robot Walking Environments Using Semi-Supervised Learning

Abstract

Convolutional neural networks trained using supervised learning can improve visual perception for human-robot walking. These advances have been possible due to largescale datasets like ExoNet and StairNet - the largest open-source image datasets of real-world walking environments. However, these datasets require vast amounts of manually annotated data, the development of which is time consuming and labor intensive. Here we present a novel semi-supervised learning system (ExoNet-SSL) that uses over 1.2 million unlabelled images from ExoNet to improve training efficiency. We developed a deep learning model based on mobile vision transformers and trained the model using semi-supervised learning for image classification. Compared to standard supervised learning (98.4%), our ExoNet-SSL system was able to maintain high prediction accuracy (98.8%) when tested on previously unseen environments, while requiring 35% fewer labelled images during training. These results show that semi-supervised learning can improve training efficiency by leveraging large amounts of unlabelled data and minimize the size requirements for manually annotated images. Future research \text{will}\text{will} focus on model deployment for onboard real-time inference and control of human-robot walking.