A Deep Reinforcement Learning based End-to-End Control Framework for Lower Limb Exoskeletons with Smooth Movement Transitions

Abstract

This paper presents an active control strategy for lower limb exoskeletons by proposing an end-to-end framework employing deep reinforcement learning (DRL) to enable smooth transitions between different movement patterns. The majority of existing methods in exoskeleton literature employ finite state machines (FSM) that have proven successful in predicting the control strategy for the next state on the basis of sensor data such as IMU data, force, etc. However, one drawback of FSM occurs due to their inflexibility regarding sudden changes. Specifically, FSM is based on clear state transitions, which makes it hard to manage smooth continuous movements and increases the chance of sudden changes in control during transitions. These, in turn, raise safety concerns for the user. While learning-based control approaches have been suggested in recent years, the validation was performed in simulation environments. Therefore, the real-world applicability remains an open research question to date. To address this issue, we provide the first contribution in this field that proposes an end-to-end learning framework with a Deep Deterministic Policy Gradient (DDPG) module to enable smooth transitions between movement patterns under real-world conditions. By introducing several evaluation metrics, we demonstrate that our framework outperforms existing methods in terms of the adaptability and smoothness in movement transitions.