Semantic segmentation of urban scenes with a location prior map using lidar measurements

Abstract

This paper proposes a method for jointly estimating road layout and segmenting urban scenes semantically by applying a relative location prior. The proposed method is based on the conjecture that some relevant elements in urban environments tend to be located in a predictable manner. This belief can be modeled as a location prior to help a segmentation algorithm infer latent labels in images more accurately. In order to apply these structural characteristics, a set of special coordinates, referred to as road-normal coordinates, are defined which are perpendicular to the road. These coordinates are determined by estimating the most fittable road layout based on a marginal probability obtained from an existing segmentation algorithm. All possible segments in an image are projected into the road-normal coordinates with the aid of depth information from sensor measurements, and the pre-trained location prior is applied to each segment as an additional potential of a conditional random field (CRF) model. The proposed method is evaluated on the publicly available KITTI dataset including images and corresponding 3D point clouds.