Driver Scanpath Prediction Based On Inverse Reinforcement Learning

Abstract

Modeling driver attention allocation by scanpath prediction plays a crucial role in advancing autonomous driving capabilities and enhancing accident anticipation. Existing studies primarily predict human scanpath for visual search, visual question answering, and free-viewing. Few studies have focused on predicting scanpath in driving scenarios. To address these limitations, we propose a novel inverse reinforcement learning-based approach through adversarial learning to effectively anticipate human-like scanpaths within different driving tasks. Particularly, we introduce a Transformer-based architecture to construct the generator and discriminator models, while integrating top-down, bottom-up, and historical information for dynamic state updated through State-Encode-with-Attention (SEA). Inspired by the human visual system, SEA adopts a fovea-like movement strategy. Experimental results on the benchmark dataset of BDD-X-diverse validate the effectiveness of our proposed method.