Motion Planning for 2-DOF Transformable Wheel Robots Using Reinforcement Learning

Abstract

Transformable robots have been developed to perform various tasks using flexible methods. However, the transformation properties present challenges in controlling and planning motion strategies, as the system model changes when transformations occur. To address this issue, we propose a planning framework based on artificial intelligence, called Geometric Manipulability Reinforcement Learning (GM-RL). GM-RL consists of two components: the manipulability estimator and the motion planner. The manipulability estimator employs graph neural networks (GNN) to provide action guidelines based on the dynamic manipulability of the transformable robots. The motion planner generates transformation plans using reinforcement learning (RL). The activation ratio <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\alpha$</tex-math></inline-formula> adjusts the ratio of the guideline accepted between the two components. In experiments utilizing a 2-DoF transformable wheel called STEP, GM-RL with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\alpha =0.5$</tex-math></inline-formula> generated an optimal transformation plan with an average dynamic manipulability measure of 0.0424, the highest measure compared to pure dynamic manipulability and reinforcement learning. A real-world experiment demonstrated that the transformation plan is efficient for overcoming stairs.