Reinforcement Learning for Stair Locomotion of a Wheeled Bipedal Robot with Contact-Guided Behavior Cloning

Abstract

This paper proposes a contact event-guided PPO with Behavior Cloning (PPO-BC) framework for stair locomotion of a 2-wheel 2-leg (2W2L) wheeled bipedal robot. Stair traversal is difficult because successful climbing depends on brief and sparse wheel-stair contact events that require precise leg lifting and posture stabilization. To address this issue, the proposed method trains a student policy using a combined objective of PPO-based reinforcement learning and behavior cloning from a pretrained frozen teacher policy. The teacher learns leg-centered climbing behaviors, while the student learns full 8-DoF control. A soft contact gate detects stair interaction directly from wheel contact forces and increases the BC contribution during critical contact phases without external terrain sensors. The method is validated under a minimal reward structure based on velocity tracking and postural stability, without stair-specific shaping rewards. Experiments in Isaac Lab simulation show that the proposed method outperforms both pure PPO and uniform PPO-BC in stair-crossing performance while maintaining stable locomotion after traversal.