Human-Like Autonomous Driving Car-Following Behavior Learning Based on Adversarial Training and Uncertainty-Aware DDPG

Abstract

Designing effective car-following (CF) models is essential for the development of safe and efficient autonomous driving systems, enabling ego vehicles to adjust their speed based on leading traffic. However, current data-driven CF models, particularly those based on reinforcement learning (RL), often face challenges in engineering reward functions and adequately addressing uncertainties in vehicle motion predictions. In response, this letter introduces a novel human-like CF policy learning approach that synergizes the Generative Adversarial Imitation Learning (GAIL) framework with an enhanced Deep Deterministic Policy Gradient (DDPG) algorithm, utilizing both human driving data and environmental interactions. We first develop a deep ensemble learning model for robust leading vehicle motion prediction with uncertainty quantification. Then, we employ GAIL to learn an implicit reward function, thereby avoiding the manual design of the reward function in DDPG. Moreover, we augment the state with predictive insights for proactive decision-making, and incorporate a dual critic strategy and behavior cloning loss to improve learning stability and mimic human-like driving. Extensive evaluations on multiple naturalistic driving datasets, complemented by thorough ablation studies, demonstrate the effectiveness of our proposed approach in achieving superior accuracy, safety, and comfort in CF tasks. The code of our method will be publicly released.