An Online Terrain Classification Framework for Legged Robots Based on Fusion of Proprioceptive and Exteroceptive Sensors

Abstract

Terrain classification is crucial for assessing terrain traversability and supporting locomotion control of legged robots. By integrating multi-source sensor information, including exteroceptive sensors and proprioceptive sensors, legged robots can acquire terrain geometric features and surface cover types. However, single-sensor approaches exhibit inherent limitations, where exteroceptive sensors are susceptible to environmental interference while proprioceptive sensors struggle to identify surface cover types. To address these challenges, this paper proposes a robust terrain classification framework that overcomes the limitations of single-modal perception through fusion of exteroceptive and proprioceptive sensors. The framework comprises a Golden Sine Optimization Algorithm-based random forest model using proprioceptive sensors to determine optimal hyperparameter combinations based on classification requirements, and a YOLOv11 network integrated with intersection over union object tracking algorithm to achieve stable image extraction during robot movement. Final terrain classification is accomplished through Kalman filter-based decision fusion. Experimental validation demonstrated classification accuracies of 94.4% for the proprioceptive module and 94.2% for the visual module in offline testing. In online fusion testing, the system achieved 95.9% overall classification accuracy, confirming the effectiveness and engineering practicality of the proposed method.