Face Relighting with Ratio Function for Explicit Geometric Representation

Abstract

This paper addresses the problem of face relighting under varying illumination conditions. Lighting is a fundamental element in portrait photography that shapes the mood, geometry, and overall realism of the captured characters. Most previous studies have mainly treated relighting as a 2D generation task without incorporating the geometric features of the characters. In contrast, inspired by ratio image-based methods, this paper proposes to disentangle shadow and brightness variations through geometric information and utilizes generative adversarial networks (GANs) to obtain relighted images with brightness consistency. We design a novel relighting-ratio function that integrates the Cook-Torrance reflectance model to more explicitly represent the face geometry than previous ratio image-based methods. This relighting-ratio function is derived from an image rendering formula that quantizes variables such as albedo that are affected by the lighting direction, while systematically excluding variables such as normal and viewpoint that are not affected by lighting. We conduct quantitative and qualitative experiments on the Multi-PIE and CelebA-HQ datasets and show that the proposed method outperforms existing SOTA methods using lighting directions.