CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering

Abstract

Intrinsic image decomposition is a long-standing, highly challenging computer vision problem, where ground truth data is very difficult to acquire. We explore the idea of using synthetic data to train CNN-based intrinsic image decomposition models, and applying these learned models to real-world images. To that end, we present CGINTRINSICS, a new, large-scale dataset of physically-based rendered images of scenes with full ground truth decompositions. The image generation process we use is carefully designed to yield high-quality, realistic images, which we find to be critical for this problem. We also propose a new end-to-end learning method that learns better decompositions by leveraging CGINTRINSICS , and optionally IIW and SAW, two recent datasets of sparse annotations on real-world images. Surprisingly, we find that a decomposition network trained solely on our synthetic data outperforms the state-of-the-art on both IIW and SAW, and performance improves even further when IIW and SAW data is added during training. Our work demonstrates the unreasonable effectiveness of carefully-rendered synthetic data for the intrinsic images task.