Reinforcement Learning-Based Adaptive Path Tracking Fusion Control Strategy for Intelligent Vehicles

Abstract

Path tracking control is a crucial technology for intelligent vehicles (IVs) to achieve accurate tracking. However, a single controller struggles to track precisely in complex environments due to respective advantages and applicable scenarios. To improve tracking performance for IVs, this letter proposes an adaptive fusion control strategy through reinforcement learning (RL), allowing multiple controllers to cooperate more efficiently in path tracking. Firstly, autonomous steering is modeled by the Pure Pursuit (PP) with foresight and the nonlinear model predictive control (NMPC) with prediction. Secondly, by integrating PP with NMPC through Proximal Policy Optimization (PPO), the control weights are dynamically adjusted to balance the contributions of both controllers. The method incorporates road information and error data as state inputs, enhancing PPO's generalization and addressing the challenge of adaptive control in complex environments. Compared to other RL methods, PPO is insensitive to hyperparameters and has a better generalizability performance. Therefore, the PPO approach results in less training time and simpler application in IV systems. Finally, a fusion control strategy that integrates PP and NMPC into the PP-PPO-NMPC framework is proposed to achieve accurate tracking and enhance the adaptability of IVs in complex scenarios. The strategy is thoroughly validated by extensive simulation and practical tests against baseline algorithms. The results confirm that the fusion PP-PPO-NMPC algorithm exhibits superior adaptability and tracking performance in complex environments.