An Actor-Critic Reinforcement Learning Scheme for Reactive 3D Optimal Motion Planning Based on Fluid Dynamics
Marios Malliaropoulos, Panagiotis Rousseas, Charalampos P. Bechlioulis, Kostas J. Kyriakopoulos
Abstract
This work proposes a novel and provably correct method for three-dimensional optimal motion planning in complex environments. Our approach models the 3D motion planning problem by solving streamlines of the potential fluid flow, filling a gap in traditional motion planning techniques by guaranteeing a closed-loop, smooth and natural-looking navigation solution. Special emphasis is given to an inherent challenge of artificial potential field (APF) methods, namely establishing proofs of safety and stability over the entire optimization process. A model-based actor-critic reinforcement learning algorithm is introduced to approximate the optimal solution to the Hamilton-Jacobi-Bellman equation and update the controller parameters in a deterministic manner. Through a series of ROS-Gazebo software-in-the-loop simulations the proposed methodology demonstrates robustness and outperforms widely used methods such as the RRT∗, highlighting its contribution to the field of 3D optimal motion planning.
BibTeX
@inproceedings{iros2024_anactorcriticrei,
title = {An Actor-Critic Reinforcement Learning Scheme for Reactive 3D Optimal Motion Planning Based on Fluid Dynamics},
author = {Marios Malliaropoulos and Panagiotis Rousseas and Charalampos P. Bechlioulis and Kostas J. Kyriakopoulos},
booktitle = {IROS 2024},
year = {2024}
}