Learning to Optimize Control Policies and Evaluate Reproduction Performance from Human Demonstrations

Abstract

We are interested in learning from demonstration (LfD) that can both learn and execute a trajectory and evaluate the quality of a previously unseen trajectory in the domain of assistive robotics. To this end, we propose a novel continuous inverse optimal control (IOC) formulation that simultaneously learns an optimal time-invariant controller and an evaluation metric from human demonstrations. We assume that the expert’s objective function is a weighted combination of physically meaningful basis objective functions. The evaluation metric is derived from the learned expert’s objective function. The benefit of this approach is twofold: 1) the controller can be optimized with respect to the learned evaluation metric and subject to the robot’s dynamic limitations and 2) the evaluation metric can evaluate the quality of a demonstrated trajectory. We validate our approach with two experiments in a robot guided therapy setting: 1) evaluating demonstrated exercises with the learned metric and 2) reproducing both unconstrained trajectories and trajectories subject to the robot’s dynamic constraints.