Object Detection with Color and Depth Images with Multi-Reduced Region Proposal Network and Multi-Pooling

Abstract

Object detection technology has received increasing research attention with recent developments in automation technology. Most studies in this field, however, use RGB images as input to deep-learning classifiers, and they rarely use depth information.So, in this paper, we use images with both RGB and depth information as input to an object detection network. We base our network on the Faster R-CNN proposed by Shih et al., and we develop a fast and accurate object detection architecture. In addition to adding depth as input, we also adjust the type of anchor boxes to improve performance on some objects. We also discuss the impact of pooling training data with multiple region proposal networks (RPN) and regions of interest (ROI).Adding depth information improved the mAP by 8.15%, from 36.86% to 45.01%, when using the SUN RGB-D dataset with 10 classes. Optimizing the anchor boxes improved the mAP from 45.01% to 45.88%. After testing various architectures with different reduced RPNs, we find that the model of 1RRPN-2ROIP performs best. The running time is 0.123 s, which is 1.8 times faster than the 3D-SSD model.