Reinforcement Learning Assist-As-Needed Control Promotes Recovery of Walking Speed Following Ankle Weight Perturbations

Abstract

Self-selected walking speed is a key outcome for exercise-based rehabilitation programs following lower-extremity trauma. This work introduces a novel reinforcement learning-based assist-as-needed (RL-AAN) controller for ankle exoskeletons, aimed at gait speed training. Built on an actor–critic architecture, the RL-AAN controller integrates a control objective that balances the trade-off between expected stride velocity (SV) errors and exoskeleton assistance. This approach allows the exoskeleton to progressively reduce ankle plantar- and dorsiflexion (PDF) assistance as the user’s performance improves, promoting active participation. The desired assistive torque is computed as the product of the actor output and the wearer’s biomechanical ankle PDF moment, estimated by a subject-agnostic model, thereby ensuring personalized and biomechanically relevant assistance. In a proof-of-concept study with healthy individuals walking on a self-paced treadmill with ankle weights, the RL-AAN controller outperformed a conventional Fixed-K controller—achieving greater immediate speed increases during assisted walking (14.2% vs. 10.0% relative to unassisted perturbed walking) and inducing short-term gait speed adaptation post-training, not observed with the conventional controller. These findings highlight the potential of RL-AAN control for subject-tailored gait training, with promising clinical implications for exercise-based rehabilitation in individuals with neurological or musculoskeletal gait impairments.