Online Design Optimization of Passive Exoskeletons Using Fast Biomechanics Simulation and Reinforcement Learning

Abstract

Exoskeletons are being adopted as assistive devices in industries such as manufacturing, logistics, and construction, aimed at reducing musculoskeletal loads in workers. Presently, their design process assumes the user to be quasi-static, optimizing the design parameters for reduction of human joint torques followed by fine-tuning through usability studies and physical prototyping. We present a method for optimizing passive exoskeleton designs before the physical prototyping stage for muscle effort reduction in dynamic tasks such as arm reaching and walking. We employ fast MuJoCo-based simulations of human biomechanics to compute the joint torques, muscle forces and muscle activations while executing task trajectories using pre-trained reinforcement learning models from the literature. We train another set of reinforcement learning models that minimize joint torques and muscle effort rates by varying the exoskeleton's design parameters online during the task motions. Baselines for comparison include the default designs of shoulder and walking assist exoskeletons from the literature, and designs obtained through conventional optimization techniques. In terms of muscle effort rates, the RL-based designs improved upon these baselines by an average of 3.42% and 1.96% respectively in the arm reaching task, and 6.28% and 5.81% in the walking task. Our method can be adapted to evaluate exoskeletons in real-time through motion capture, and for muscle-aware online control of powered exoskeletons.