Spatiotemporal Learning of Directional Uncertainty in Urban Environments With Kernel Recurrent Mixture Density Networks

Abstract

Autonomous vehicles operating in urban environments need to deal with an abundance of other dynamic objects, such as pedestrians and vehicles. This requires the development of predictive models that capture the complexity and long-term patterns of motion in the environment. We approach this problem by modeling movement directions of a typical object in the environment over time. We propose a method to build a continuous map to capture spatiotemporal movements. At a given coordinate in space and time, our method provides a multi-modal probability density function over the possible directions an object can move in. We achieve this by projecting data into a high-dimensional feature space using sparse approximate kernels, and passing the highdimensional features through a long short-term memory (LSTM) network. A mixture density network (MDN) with von Mises distributions is then trained on the hidden representations of the LSTM. Once trained, the outputs of the MDN can be used to construct the probability distribution over movement directions. The model is continuous in both spatial and temporal domains, as we can query the change of directional uncertainty at arbitrary resolution in space and time. We conduct experiments on three simulated and three real-world benchmark datasets with pedestrians and vehicles, and demonstrate that both the continuous nature and the addition of temporal patterns improves performance.