Learning to traverse challenging terrain using vision and forward kinematics

Abstract

In this letter, we propose a new method for visual locomotion controller in quadruped robots, aimed at enhancing their capability to traverse challenging terrain. Our approach integrates computer vision techniques with robust locomotion control to improve terrain traversal. To facilitate terrain perception, we use onboard cameras and body sensors to collect real-world visual and proprioceptor data, and utilize forward kinematics to convert joint angles into precise foot positions. This enables accurate estimation of terrain height, which serves as supervised training data for our visual motion controller. This integrated approach improves the robot's ability to anticipate and adapt to diverse terrain conditions, potentially advancing quadruped locomotion in unstructured environments. Our model is deployed on A1 robot from Unitree. Experimental results show that our proposed method can enable the robot to stably climb stairs and pass through sand, grass, snow, and uneven roads.