LNLFace: Enhanced Blind Face Restoration With Local and Non-local Lookups

Abstract

Existing reference-based blind face restoration (BFR) methods tend to either focus on the detailed texture of facial components or only conduct code prediction in the latent space, often neglecting their complementary relationship. To deal with it, the integration of local and non-local lookups that interact to ensure both fine-grained details and global geometric consistency is an eminently practical yet challenging solution. Specifically, Facial Component Dictionaries and High-Quality Feature Codebooks, which are pre-constructed from a large corpus of high-quality face images, perform their own functions of low- and high-level features. Thus, we first introduce an xLSTM-based Degradation-Aware Module (DAM) to mitigate code prediction biases suffering from degradation, then develop several two-stage Global Semantic Attention Modules (GSAM) to refine the multi-scale local details by the non-local insights. Finally, our proposed method, termed Local and Non-local Lookups for BFR (LNLFace), has demonstrated comparable or even superior performance than state-of-the-art methods, on both synthetic and real-world datasets. The codes are available at: https://github.com/yanwd628/LNLFace.