Learning to Rock-and-Walk: Dynamic, Non-Prehensile, and Underactuated Object Locomotion Through Reinforcement Learning

Abstract

When moving objects that are too bulky or heavy to be grasped or lifted, robotic manipulation can benefit from the object's interaction with the support surface and its natural dynamics under gravity. In this work, we show that such dynamic, underactuated manipulation capability can be acquired through reinforcement learning and deployed on real robot systems. First, we present a framework to learn a control policy for object transport in a dynamic simulation environment, featuring the object and the support surface. We then demonstrate successful object locomotion with the learned policy through a set of simulated and real-world experiments, performed with a robot arm and an aerial robot interacting with the object in a non-prehensile manner. While the object, which is in contact with the support surface, oscillates sideways passively under gravity, the robot uses the learned policy to move the object forward with a steady gait by regulating the mechanical energy and the posture of the object. Our experiment results show that the learned policy can transport the object through unmodeled effects of terrain and perturbation.