A 3D Convolutional Neural Network Towards Real-Time Amodal 3D Object Detection

Abstract

We focus on the task of amodal 3D object detection, which is to predict object locations, dimensions, poses and categories in the real world. We introduce a 3D Convolutional Neural Network that takes a volumetric representation of an indoor scene as input and predicts 3D object bounding boxes, object categories, and orientations. Unlike prior state-of-the-arts, our approach does not depend on region proposal techniques to hypothesize object locations. We treat detection and recognition as one regression problem in a single network. Our elegant model is extremely fast and all predictions are reasoned from the global context of a point cloud in a continuous pipeline. We evaluate our approach on two standard datasets: the NYUv2 RGBD dataset and the SUN RGBD dataset. Experiments show that our approach is faster than start-of-the-art 3D detectors by several orders of magnitude towards real-time amodal 3D object detection.