Receding Horizon Reinforcement Learning with Autoregressive Model for Motion Control of Autonomous Vehicles

Abstract

This paper presents a model-based reinforcement learning (MBRL) approach with a receding horizon mechanism to optimize the lateral trajectory-tracking performance of autonomous vehicles (AVs). Accurate modeling of complex vehicle dynamics and adaptation to dynamic environments with limited data pose significant challenges for MBRL in AV control. To address these challenges, we propose sample-efficient algorithms that leverage autoregressive modeling to adapt from limited data while managing complex vehicle dynamics. Unlike traditional methods reliant on fixed models, our approach uses the temporal reasoning of autoregressive (AR) models to compensate for the residual dynamics, which effectively approximates the local effects of nonlinearities and disturbances. Integrated with real-time sensor data, the residual generation model is continuously refined via incremental learning in a closed-loop framework, enhancing adaptability. This architecture, combining physical modeling with data-driven residuals, maintains interpretability and improves responsiveness in complex scenarios. CarSim simulations demonstrate superior performance over other state-of-the-art learning-based predictive controllers and classical methods for AV lateral control. Real-world validation on a HongQi electric vehicle (HQEV) confirms the algorithm’s effectiveness, showing significant improvements over classical model predictive control (MPC). This approach holds substantial potential for advanced driver-assistance systems (ADAS) and fully autonomous driving, enabling precise control under diverse conditions.