BPL: Generalizable Deepfake Detection via Bias-only Pair-aware Learning

Abstract

The detection of synthetic images has traditionally been framed as a binary classification problem. However, we argue that this formulation overlooks a fundamental structural property of generative datasets: synthetic images are not independent samples, but are implicitly paired with real images sharing the same semantic source. Existing methods treat real and fake images as independent instances, failing to capture generation-induced relational discrepancies in real–fake pairs. Moreover, models tend to rapidly overfit to seen fake patterns, leading to poor generalization to unseen ones. To overcome these challenges, we propose a novel detection framework that explicitly mines real–fake pairs by constructing source-guided mappings or leveraging nearest-neighbor relationships in the CLIP embedding space. We then introduce pair-wise discrepancy learning that explicitly enlarges generation-induced deviations and discrepancy inversion to mitigate overfitting. Moreover, to preserve pretrained semantic representations while improving generalization, we adopt a bias-only fine-tuning scheme that restricts model capacity during adaptation. Extensive experiments show that our approach achieves superior generalization across unseen fake patterns.