Action Space Design in Reinforcement Learning for Robot Motor Skills
Julian Eßer, Gabriel B. Margolis, Oliver Urbann, Sören Kerner, Pulkit Agrawal
Abstract
Practitioners often rely on intuition to select action spaces for learning. The choice can substantially impact final performance even when choosing among configuration-space representations such as joint position, velocity, and torque commands. We examine action space selection considering a wheeled-legged robot, a quadruped robot, and a simulated suite of locomotion, manipulation, and control tasks. We analyze the mechanisms by which action space can improve performance and conclude that the action space can influence learning performance substantially in a task-dependent way. Moreover, we find that much of the practical impact of action space selection on learning dynamics can be explained by improved policy initialization and behavior between timesteps.
BibTeX
@inproceedings{
e{\ss}er2024action,
title={Action Space Design in Reinforcement Learning for Robot Motor Skills},
author={Julian E{\ss}er and Gabriel B. Margolis and Oliver Urbann and S{\"o}ren Kerner and Pulkit Agrawal},
booktitle={8th Annual Conference on Robot Learning},
year={2024},
url={https://openreview.net/forum?id=GGuNkjQSrk}
}