Online Policy Adaptation for Personalized Lane-Keeping via Driver Intervention Guided Reinforcement Learning

Abstract

Learning-based online adaptive driving policies hold considerable promise for enabling human-preferred autonomous driving or advanced driver assistance systems. However, existing personalization approaches often rely on limited style definitions or behavior cloning, requiring extensive data and placing cognitive burdens on users. To address these limitations, an Online Personalized Policy Adaptation (OPPA) framework is proposed based on direct driver interventions, where real-time corrective actions on the steering wheel and pedals are treated as implicit feedback reflecting individual driving preferences. A hybrid learning strategy integrates reinforcement learning with intervention-guided updates, enabling efficient online adaptation. To mitigate overfitting and prevent catastrophic forgetting during long-term personalization, the Memory Aware Synapses (MAS) regularization technique is selectively activated based on detected shifts in driver preferences. The framework is initialized from a generalized policy pretrained using the TD3 algorithm and a composite reward function that captures perceived risk, comfort, and efficiency. Driver-in-the-loop experiments incorporating subjective evaluations are conducted with 21 drivers across four representative curved-lane scenarios to assess the effectiveness of the personalization. Results demonstrate that the proposed framework achieves rapid and effective personalization, typically needing fewer than two intervention episodes. Subjective satisfaction improves by 18.9% over the baseline. The average adaptation time per intervention ranges from 0.75 to 1.04 seconds, depending on MAS activation, confirming the effectiveness and responsiveness of the OPPA framework in aligning driving policies with individual preferences.