Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image

Abstract

We propose a material acquisition system that can recover the spatially-varying BRDF and normal map of a near-planar surface from a single image captured by a handheld mobile phone camera. Our technique images the surface under arbitrary environment lighting with the flash turned on, thereby avoiding shadows while simultaneously capturing high-frequency specular highlights. We train a CNN to regress an SVBRDF and surface normals from this image. Our network is trained using a large-scale SVBRDF dataset and designed to incorporate physical insights for material estimation, including an in-network rendering layer to model appearance and a material classification task to provide additional supervision during training. Finally, we refine the results from the network using a dense CRF module whose terms are designed specifically for our task. We demonstrate that our CNN-based SVBRDF inference leads to state-of-the-art results on a wide variety of materials on both synthetic and real data. We also provide extensive ablation studies to evaluate our network and demonstrate large improvements in comparisons with prior works.