Knowledge Reasoning for Semantic Segmentation

Abstract

The convolution operation suffers from a limited receptive field, while global modeling is fundamental to dense prediction tasks, such as semantic segmentation. However, most existing methods treat the recognition of each region separately and overlook crucial global semantic relations between regions in one scene. These methods cannot segment the semantic regions accurately due to the lack of global-level supervision or guidance of external knowledge. To overcome the limitation of the traditional method, we propose a Knowledge Reasoning Net (KRNet) that consists of two crucial modules: (1) a prior knowledge mapping module that incorporates external knowledge by graph convolutional network to guide learning semantic representations and (2) a knowledge reasoning module that correlates these representations with a graph built on the external knowledge and explores their interactions via the knowledge reasoning. Experiments on Cityscapes and ADE datasets demonstrate the effectiveness of our proposed methods on semantic segmentation.