A Hierarchical Framework for Quadruped Locomotion Based on Reinforcement Learning

Abstract

Quadruped locomotion is a challenging task for learning-based algorithms. It requires tedious manual tuning and is difficult to deploy in reality due to the reality gap. In this paper, we propose a quadruped robot learning system for agile locomotion which does not require any pre-training and works well in various real-world terrains. We introduce a hierarchical learning framework that uses reinforcement learning as the high-level policy to adjust the low-level trajectory generator for better adaptability to the terrain. We compact the observation and action space of the reinforcement learning to deploy it on a host computer in reality. Besides, we design a trajectory generator guided by robot posture, which can generate adaptive foot trajectory to interact with the environment. Experimental results show that our system can be easily deployed in reality while only trained in simulation, and also has the advantages of fast convergence and good terrain adaptability. The supplementary video demonstration is available at https://vsislab.github.io/hfql/.