Personalizing Human Gait Entrainment: A Reinforcement Learning Approach to Optimizing Magnitude of Periodic Mechanical Perturbations

Abstract

The feasibility of gait entrainment to periodic mechanical perturbations varies with perturbation magnitude in neurotypical individuals. Effective design of gait entrainment studies thus requires a systematic approach to personalize periodic perturbation parameters. However, current studies still rely on manually selecting the perturbation magnitude, a practice that is neither efficient nor optimal for individual users. This study proposes a new reinforcement learning (RL) method to personalize the minimum magnitude of periodic perturbation to hip flexion that ensures successful entrainment. The method entails offline learning and in situ adaptation (OLAP), where offline learning involves training a deep Q-network (DQN), which is subsequently used in situ to guide the adaptive selection of an optimal perturbation magnitude for individuals. This study recruited thirteen healthy participants, with entrainment characteristics data from seven participants used for offline DQN training. The remaining six participants performed in situ adaptation to identify their personalized optimal perturbation parameters. Results demonstrate that the OLAP agent effectively tailored a minimum perturbation magnitude for each of the six participants in the adaptation group, leveraging generalization from the DQN policy. All adaptation group participants achieved a 100% entrainment success rate at their personalized perturbation magnitude during a 3-trial post-evaluation session, highlighting the agent's effectiveness. The efficiency and robustness of our approach underscore its significance in designing future optimal gait entrainment studies for diverse population groups.