Unsupervised Learning of Facial Optical Flow via Occlusion-Aware Global-Local Matching

Abstract

Estimating optical flow from facial videos is an essential preprocessing step for many applications. However, it is a challenging task as the facial videos contain rich expressions, large displacements, and complex occlusions. Obtaining the ground truth optical flow for facial videos is very difficult, which hinders the supervised learning of optical flow from monocular in-the-wild facial videos. In this paper, we provide an effective and accurate method for the unsupervised learning of optical flow from facial videos. An occlusion-aware global-local matching model is introduced for the joint reasoning of optical flow and occlusions. We propose a novel occlusion estimation paradigm to detect occlusions caused by facial expressions and pose variations. Experiments demonstrate that our method compares favorably against the state-of-the-art methods in facial optical flow estimation.