Learning Oscillator-Based Gait Controller for String-Form Soft Robots Using Parameter-Exploring Policy Gradients

Abstract

This paper presents a methodology to design mechanosensor feedback to oscillator-based controller for worm-like soft-bodied robots. A reinforcement learning technique, i.e., PEPG, is employed to embed appropriate mechanosensor feedback to harness global entrainment among the controller, the body dynamics, and the environment without explicitly designing the interaction between the oscillators. Another reinforcement learning, actor-critic, was applied to train the controller for the simulation models to analyze the effectiveness of PEPG in the system. Furthermore, the gait controller was trained under different body dynamics, i.e., the physical model of a caterpillar and an earthworm. We found that PEPG is suitable for the system probably because it does not add exploration noise to actions and it conducts episode based parameter updates. The simulation results show the proposed method can acquire distinct behavior, i.e., caterpillars' crawling, inching and earthworms' crawling, under different body dynamics. The outcome implies, that by utilizing appropriate learning method, desired functionality can be achieved in soft-bodied robots without explicitly designing their behavior.