Semi-Supervised Learning: Structure, Reflectance and Lighting Estimation From a Night Image Pair

Abstract

While unsupervised approaches have been proposed to estimate reflectance and shading layers for images, the decomposition process is a challenging, under-determined inverse problem. Previous unsupervised approaches for intrinsic decomposition of images are strictly reliant on special image sequences, for example, image sequences for a fixed scene with changing illumination or multi-view images containing rich illumination variation. As an alternative to these unsupervised intrinsic decomposition methods, we propose a semi-supervised method. Our method adopts a simplified scene representation to greatly reduce the complexity of spatially varying lighting, which allows us to partially restore the lighting from a night image pair. Moreover, we design several novel unsupervised guiding losses, and the training data used in our method are easily collected by setting different exposure times for a standard stereo setup. We further demonstrate the effectiveness of the proposed method by quantitatively and qualitatively comparing our method with recent works on the modified MPI dataset and a collected highly dynamic dataset named NightCampus. The NightCampus dataset has been released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/kewangtt/srlefnip</uri> .