Adaptive Assist-as-needed Control Based on Actor-Critic Reinforcement Learning
Yufeng Zhang, Shuai Li, Karen J. Nolan, Damiano Zanotto
Abstract
In robot-assisted rehabilitation, assist-as-needed (AAN) controllers have been proposed to promote subjects’ active participation, which is thought to lead to better training outcomes. Most of these AAN controllers require a patient-specific manual tuning of the parameters defining the underlying force-field, which typically results in a tedious and time-consuming process. In this paper, we propose a reinforcement-learning-based impedance controller that actively reshapes the stiffness of the force-field to the subject’s performance, while providing assistance only when needed. This adaptability is made possible by correlating the subject’s most recent performance to the ultimate control objective in real-time. In addition, the proposed controller is built upon action dependent heuristic dynamic programming using the actor-critic structure, and therefore does not require prior knowledge of the system model. The controller is experimentally validated with healthy subjects through a simulated ankle mobilization training session using a powered ankle-foot orthosis.
BibTeX
@inproceedings{iros2019_adaptiveassistas,
title = {Adaptive Assist-as-needed Control Based on Actor-Critic Reinforcement Learning},
author = {Yufeng Zhang and Shuai Li and Karen J. Nolan and Damiano Zanotto},
booktitle = {IROS 2019},
year = {2019}
}