Patch-Discontinuity Mining for Generalized Deepfake Detection

TL;DR

GenDF introduces a parameter-efficient framework that adapts large-scale vision models to deepfake detection by learning patch-level continuity in real faces and discontinuities in forged ones. It integrates Deepfake-Specific Representations Learning (DSRL) with low-rank ViT fine-tuning, Feature Space Redistribution (FSR) to sharpen class separation, and Class-Invariant Feature Augmentation (CIFAug) to expand variability without extra parameters, all optimized via a composite loss. The approach achieves state-of-the-art generalization in cross-domain and cross-manipulation settings while using only about 0.28M trainable parameters, and it remains efficient in inference. These findings demonstrate that a carefully designed, compact fine-tuning strategy can leverage large pretrained models for robust, scalable deepfake detection across diverse scenarios.

Abstract

The rapid advancement of generative artificial intelligence has enabled the creation of highly realistic fake facial images, posing serious threats to personal privacy and the integrity of online information. Existing deepfake detection methods often rely on handcrafted forensic cues and complex architectures, achieving strong performance in intra-domain settings but suffering significant degradation when confronted with unseen forgery patterns. In this paper, we propose GenDF, a simple yet effective framework that transfers a powerful large-scale vision model to the deepfake detection task with a compact and neat network design. GenDF incorporates deepfake-specific representation learning to capture discriminative patterns between real and fake facial images, feature space redistribution to mitigate distribution mismatch, and a classification-invariant feature augmentation strategy to enhance generalization without introducing additional trainable parameters. Extensive experiments demonstrate that GenDF achieves state-of-the-art generalization performance in cross-domain and cross-manipulation settings while requiring only 0.28M trainable parameters, validating the effectiveness and efficiency of the proposed framework.

Figures (4)

• Figure 1: Comparison of existing deepfake detection methods and our method in terms of the number of trainable parameters and intra-domain and cross-domain performance. The x-axis denotes the AUC results with training and testing on FF++(HQ), while the y-axis represents the AUC results with training on FF++(HQ) and testing on the Celeb-DF dataset. The size of the circles indicates the scale of trainable parameters. Our framework achieves the best generalization performance among all methods with the fewest trainable parameters.
• Figure 2: The overall pipeline of the proposed generalizable deepfake detection framework (GenDF). The facial images (real or fake) first enter a ViT backbone for representation embedding in a low-dimensional space. Then, we optimize the distributions to learn more discriminative features. Next, these two kinds of features go through the class-invariant feature augmentation procedure to improve the generalization abilities of our method.
• Figure 3: The Grad-CAM activation map on the FF++(HQ) dataset.
• Figure 4: The t-SNE feature distribution visualization of real and forged facial images obtained by the baseline model (ViT) and our proposed method (Ours+ViT) under the intra-testing and the cross-testing settings. (a) ViT on the intra-testing dataset (FF++). (b) Ours+ViT on the intra-testing dataset (FF++). (c) Ours+ViT on the cross-testing dataset (DFD). (d) Ours+ViT without Class Invariant Augmentation (CIFAug) function on the cross-testing dataset (DFD).