Simultaneous Prediction of Pedestrian Trajectory and Actions based on Context Information Iterative Reasoning

Abstract

Pedestrian trajectories and actions prediction in complex environment is challenging due to the complexity of human behavior and a variety of internal and external stimuli. Much works has gone towards predicting trajectories and actions separately without mining the coupling relationships between them, which is an important information for our humans to reason and predict. Inspired by this, we propose an end-to-end joint context information iterative reasoning network (CIR-Net). Specifically, a novel heterogeneous spatiotemporal graph module (HST-Graph) is proposed to encode and aggregate multiple types of context information of the motion pattern and the scene. And an action-trajectory hybrid guidance module is proposed to enhance the ability of long-time prediction by utilizing the internal coupling between actions and trajectory. Moreover, an iterative reasoning structure is designed to iteratively correcting the trajectory and actions prediction error. Experimental results on the ETH&UCY and VIRAT datasets demonstrate the favorable performance of the framework.