Learning Real Facial Concepts for Independent Deepfake Detection

TL;DR

This work tackles the generalization gap in deepfake detection by shifting focus from sole reliance on forgery artifacts to learning a comprehensive concept of real faces. It introduces RealID, comprising Real Concept Capture (RealC2) with a Multi-Real Memory of real prototypes and an Independent Dual-Decision (IDC) classifier that decouples real-face evaluation from forgery cues. The model uses Prototype Distinction and Prototype Diversity losses to cultivate diverse, discriminative real representations and employs an augmented four-way output with a regularization term to reduce misguidance from artefacts. Across five cross-dataset benchmarks, RealID achieves notable improvements in average AUC and demonstrates robustness across backbones, indicating improved generalization for real-face recognition in unseen domains.

Abstract

Deepfake detection models often struggle with generalization to unseen datasets, manifesting as misclassifying real instances as fake in target domains. This is primarily due to an overreliance on forgery artifacts and a limited understanding of real faces. To address this challenge, we propose a novel approach RealID to enhance generalization by learning a comprehensive concept of real faces while assessing the probabilities of belonging to the real and fake classes independently. RealID comprises two key modules: the Real Concept Capture Module (RealC2) and the Independent Dual-Decision Classifier (IDC). With the assistance of a MultiReal Memory, RealC2 maintains various prototypes for real faces, allowing the model to capture a comprehensive concept of real class. Meanwhile, IDC redefines the classification strategy by making independent decisions based on the concept of the real class and the presence of forgery artifacts. Through the combined effect of the above modules, the influence of forgery-irrelevant patterns is alleviated, and extensive experiments on five widely used datasets demonstrate that RealID significantly outperforms existing state-of-the-art methods, achieving a 1.74% improvement in average accuracy.

Figures (6)

• Figure 1: (a) The real samples are misclassified as fake (highlighted in red), due to the local imperfections that are not actual forgery traces; (b) The features of real faces tend to cluster more tightly; (c) Non-overlapping test samples leading to potential misclassification.
• Figure 2: Overall architecture of the RealID framework. RealID consists of two main modules: (i) the Real Concept Capture($\mathrm{RealC^2}$) module, which uses a multi-real memory mechanism to learn a more comprehensive real facial concept, and (ii) the Independent Dual-Decision(IDC) module, which leverages regularization terms to independently optimize the decision-making process for different categories.
• Figure 3: Illustration of the update process for real facial prototypes.
• Figure 4: AUC (%) comparison under different hyperparameter combinations. For $\lambda_1$, $\lambda_1$, and $\lambda_1$, we vary one of their values while keeping the other two values fixed at 0.5.
• Figure 5: Illustration for no-cherry-pick heatmaps from four datasets, comparing our RealID with the SoTA baseline SBI.