Facial Texure Perceiver: Towards High-Fidelity Facial Texture Recovery with Input-Level Inductive Biased Perceiver IO

Abstract

This paper presents a new method, called Facial Texture Perceiver. It deals with the task of facial texture recovery from in-the-wild images without 3D supervision. Motivated by their success in various computer vision tasks, we attempt to use transformers for this task. However, capturing high-fidelity facial details requires a large number of mesh vertices and in this case, naively applying vanilla transformer can incur prohibitively high computational and memory costs. We address this challenge by mapping the input with a large number of mesh vertices to a latent space and performing their attention on this space. Also, we introduce input-level inductive biases by injecting the geometry and appearance embeddings as extra inputs. It helps to data-efficiently learn and generalize in-the-wild domains. The resulting architecture enable the application of Transformers to high-resolution facial meshes. Experiments on CelebA, MICC-Florence and MoFA-test datasets demonstrate that our method can accurately reconstruct facial textures, outperforming state-of-the-art methods.