Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Open-world Generalized Deepfake Detection: General Feature Extraction via Unsupervised Domain Adaptation

Midou Guo, Qilin Yin, Wei Lu, Xiangyang Luo

TL;DR

The paper tackles open-world deepfake detection where unlabeled target data far exceeds limited labeled data. It introduces OWG-DS, a strategy that combines Domain Distance Optimization (DDO), Similarity-based Class Boundary Separation (SCBS), and an Adversarial Domain Classification (ADC) module to align cross-domain features and sharpen class boundaries, optimized with a multi-term loss that includes $\mathcal{L}_{CE}$, $\mathcal{L}_{DAL}$, $\mathcal{L}_{SCBS}$, $\mathcal{L}_{adv}$, and KL regularization. The approach trains by pretraining on a labeled source domain and then adapting to a large unlabeled target domain in a shared feature space, using global centroids and positive-pair learning to mitigate domain shift. Experimental results on FF++ family, Celeb-DF, and DFDC demonstrate improved cross-manipulation and cross-dataset performance, data efficiency with limited target data, and model-agnostic applicability, indicating strong generalization to unseen forgery methods. Overall, the work advances practical open-world deepfake detection by enabling robust detection with limited labeled data and abundant unlabeled data, through principled domain alignment and boundary refinement.

Abstract

With the development of generative artificial intelligence, new forgery methods are rapidly emerging. Social platforms are flooded with vast amounts of unlabeled synthetic data and authentic data, making it increasingly challenging to distinguish real from fake. Due to the lack of labels, existing supervised detection methods struggle to effectively address the detection of unknown deepfake methods. Moreover, in open world scenarios, the amount of unlabeled data greatly exceeds that of labeled data. Therefore, we define a new deepfake detection generalization task which focuses on how to achieve efficient detection of large amounts of unlabeled data based on limited labeled data to simulate a open world scenario. To solve the above mentioned task, we propose a novel Open-World Deepfake Detection Generalization Enhancement Training Strategy (OWG-DS) to improve the generalization ability of existing methods. Our approach aims to transfer deepfake detection knowledge from a small amount of labeled source domain data to large-scale unlabeled target domain data. Specifically, we introduce the Domain Distance Optimization (DDO) module to align different domain features by optimizing both inter-domain and intra-domain distances. Additionally, the Similarity-based Class Boundary Separation (SCBS) module is used to enhance the aggregation of similar samples to ensure clearer class boundaries, while an adversarial training mechanism is adopted to learn the domain-invariant features. Extensive experiments show that the proposed deepfake detection generalization enhancement training strategy excels in cross-method and cross-dataset scenarios, improving the model's generalization.

Towards Open-world Generalized Deepfake Detection: General Feature Extraction via Unsupervised Domain Adaptation

TL;DR

The paper tackles open-world deepfake detection where unlabeled target data far exceeds limited labeled data. It introduces OWG-DS, a strategy that combines Domain Distance Optimization (DDO), Similarity-based Class Boundary Separation (SCBS), and an Adversarial Domain Classification (ADC) module to align cross-domain features and sharpen class boundaries, optimized with a multi-term loss that includes , , , , and KL regularization. The approach trains by pretraining on a labeled source domain and then adapting to a large unlabeled target domain in a shared feature space, using global centroids and positive-pair learning to mitigate domain shift. Experimental results on FF++ family, Celeb-DF, and DFDC demonstrate improved cross-manipulation and cross-dataset performance, data efficiency with limited target data, and model-agnostic applicability, indicating strong generalization to unseen forgery methods. Overall, the work advances practical open-world deepfake detection by enabling robust detection with limited labeled data and abundant unlabeled data, through principled domain alignment and boundary refinement.

Abstract

With the development of generative artificial intelligence, new forgery methods are rapidly emerging. Social platforms are flooded with vast amounts of unlabeled synthetic data and authentic data, making it increasingly challenging to distinguish real from fake. Due to the lack of labels, existing supervised detection methods struggle to effectively address the detection of unknown deepfake methods. Moreover, in open world scenarios, the amount of unlabeled data greatly exceeds that of labeled data. Therefore, we define a new deepfake detection generalization task which focuses on how to achieve efficient detection of large amounts of unlabeled data based on limited labeled data to simulate a open world scenario. To solve the above mentioned task, we propose a novel Open-World Deepfake Detection Generalization Enhancement Training Strategy (OWG-DS) to improve the generalization ability of existing methods. Our approach aims to transfer deepfake detection knowledge from a small amount of labeled source domain data to large-scale unlabeled target domain data. Specifically, we introduce the Domain Distance Optimization (DDO) module to align different domain features by optimizing both inter-domain and intra-domain distances. Additionally, the Similarity-based Class Boundary Separation (SCBS) module is used to enhance the aggregation of similar samples to ensure clearer class boundaries, while an adversarial training mechanism is adopted to learn the domain-invariant features. Extensive experiments show that the proposed deepfake detection generalization enhancement training strategy excels in cross-method and cross-dataset scenarios, improving the model's generalization.
Paper Structure (18 sections, 12 equations, 7 figures, 4 tables)

This paper contains 18 sections, 12 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Task Definition
  5. Overview
  6. Domain Distance Optimization
  7. Similarity-based Class Boundary Separation
  8. Adversarial Domain Classification
  9. Loss Functions
  10. Experiments
  11. Experimental Setting
  12. Cross manipulation methods
  13. Cross manipulation datasets
  14. Ablation Study
  15. Effectiveness of different components:
  16. ...and 3 more sections

Figures (7)

  • Figure 1: Traditional deepfake detection
  • Figure 2: Traditional domain transfer learning
  • Figure 3: Open-world unsupervised deepfake detection
  • Figure 5: The framework of the Open-World Generalization Enhancement Training Strategy (OWG-DS) includes the Domain Distance Optimization (DDO) module, the Similarity-Based Class Boundary Separation (SCBS) module, and the adversarial training mechanism. $F'_S$ and $F'_T$ are features that are fully aligned after training.
  • Figure 6: The Domain Distance Optimization (DDO) diagram shows the dynamic process of cross-domain feature alignment by comparing the feature distribution before and after optimization. The left side (before optimization) displays the original distribution of the source domain (blue cluster) and target domain (pink cluster), with a noticeable shift between their centroids (red triangles). The right side (after optimization) shows the feature space after processing by DDO, where the centroids move closer together, and the samples spread out.
  • ...and 2 more figures