A Skill-Based Hierarchical Framework with Dangerous Action Masking for Autonomous Navigation of Jumping Robots

Abstract

Achieving autonomous navigation for biologically inspired jumping robots remains a long-standing challenge, due to the inherent instability of jumping motions and the limitations in onboard sensor capabilities. This paper proposes a skill-based hierarchical framework with dangerous action masking (SH-DAM) for autonomous navigation of jumping robot. The framework, based on hierarchical reinforcement learning, includes a low-level controller that learns locomotion skills (crawling, turning and jumping) to overcome various obstacles. A high-level controller selects and coordinates these skills, while also incorporating curriculum learning to enhance the performance of navigation tasks. For safe navigation, we utilize dangerous action masking to suppress the probability of selecting jump motions in dangerous regions. We improved the locust-inspired jumping robot platform JumpBot-S, by integrating a lightweight time-of-flight (ToF) sensor, and constructed a range of complex environments for experiments. Simulation results demonstrate that SH-DAM enables the robot to autonomously complete challenging navigation tasks. Compared to baseline algorithms, our method achieves a 12.57% increase in success rate, a 55.88% reduction in stuck rate, and a 57.89% reduction in rollover rate. Finally, we deployed our framework in real-world environments and conducted experiments in both normal lit and dimly lit conditions. This framework provides a new paradigm for jumping robot navigation in complex environments.