Personalized Reinforcement Learning Control of Soft Robotic Exosuit for Assisting Human Normative Walking with Reduced Effort

Abstract

Wearable lower limb robots are promising technologies to assist human locomotion. Soft robotic exosuits introduce a promising solution for reducing muscle effort and metabolic cost as they are lightweight, transparent and inherently safe. However, it is challenging to effectively control such soft robots and personalize the assistance for individual users. With the difficulty in developing robust dynamic model of the human-soft robot system, especially the interacting dynamics between the human and the robot, traditional control methods have seen limited success in addressing these challenges. Reinforcement learning (RL), a data-driven optimal control method, provides a naturally promising alternative. In this study, we propose an innovative control design approach to enable human normative walking with reduced physical effort. To achieve this goal, we propose to first offline learn an exosuit controller for typical human normative walking which is then used in the online phase of control tuning for individual users. Four participants are recruited to test the exosuit controller in treadmill walking. Our results show that online tuning for individual users reaches convergence quickly, typically in one experimental trial due to using an efficient offline pre-trained policy. Furthermore, the RL control of the exosuit results in an average muscle effort reduction of 8.8% and 2.8% for the vastus lateralis and biceps femoris as measured by electromyography (EMG) sensors. These results provide the first evidence of customizing the soft exosuit assistance for individual users.