PreSim: A 3D Photo-Realistic Environment Simulator for Visual AI

Abstract

Recent years have witnessed great advancement in visual artificial intelligence (AI) research based on deep learning. To take advantage of deep learning, we need to collect a large amount of data in various environments and conditions. However, collecting such data is time-consuming and labor-intensive. Apart from that, developing and testing visual AI algorithms for multisensory models is expensive and in some cases dangerous processes in the real world. We present PreSim, a 3D environment simulator which provides photo-realistic simulations using a view synthesis module and supports flexible configuration of multimodal sensors to address both of these issues. For our view synthesis module we introduce novel depth refinement, adaptive view selection and layered rendering, to provide realistic imagery. We demonstrate that PreSim has several advantages: (i) it provides a photo-realistic 3D environment which allows seamlessly integrating multisensory models in the virtual world and enables them to perceive and navigate scenes, (ii) it has an internal view synthesis module which allows transforming algorithms developed and tested in simulation to physical platforms without domain adaption, (iii) it can generate a large amount of data for vision-based applications, such as depth estimation and object pose estimation.