CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition

TL;DR

This work proposes a Deep Information Decomposition (DID) framework to improve Cross-dataset Deepfake Detection (CrossDF) and introduces an adversarial mutual information minimization strategy that enhances the separability between these two types of information through decorrelation learning.

Abstract

Deepfake technology poses a significant threat to security and social trust. Although existing detection methods have shown high performance in identifying forgeries within datasets that use the same deepfake techniques for both training and testing, they suffer from sharp performance degradation when faced with cross-dataset scenarios where unseen deepfake techniques are tested. To address this challenge, we propose a Deep Information Decomposition (DID) framework to enhance the performance of Cross-dataset Deepfake Detection (CrossDF). Unlike most existing deepfake detection methods, our framework prioritizes high-level semantic features over specific visual artifacts. Specifically, it adaptively decomposes facial features into deepfake-related and irrelevant information, only using the intrinsic deepfake-related information for real/fake discrimination. Moreover, it optimizes these two kinds of information to be independent with a de-correlation learning module, thereby enhancing the model's robustness against various irrelevant information changes and generalization ability to unseen forgery methods. Our extensive experimental evaluation and comparison with existing state-of-the-art detection methods validate the effectiveness and superiority of the DID framework on cross-dataset deepfake detection.

Figures (7)

• Figure 1: Various information changes entangled with the deepfake information in traditional methods (top) would affect real/fake classification accuracy, leading to a sharp degradation in performance when the discrepancies in these components between the training and test sets are more significant than the differences between real and deepfake information. Our deep information decomposition (DID) method (bottom) separates the deepfake information from various information irrelevant to real/fake classification to improve the robustness of deepfake detection.
• Figure 2: Overview of our Deep Information Decomposition (DID) framework. The feature map $I_X$ of an input face image $X$ from a backbone network $G$ is decomposed into deepfake information $I_{df}$ and non-deepfake information $I_{os}$ adaptively under the guidance of the deepfake attention network $A_{df}$ and the supervision of the deepfake classification module. The domain attention network $A_{dom}$ and the domain classification module capture the forgery method-related (domain) information $I_{dom}$ and ensure that $I_{dom}$ is included in the non-deepfake information but absent in the deepfake information. In addition, the decorrelation learning module ensures no overlapping between deepfake information and non-deepfake information. This module consists of an information estimation network $T$, which functions in a max-min manner with the information decomposition module through the gradient reversal layer (GRL). $C$ and $\overline{C}$ are the deepfake and domain classifiers, respectively.
• Figure 3: Architecture of the deepfake (domain) attention network. This network takes the face information $I_X$ (deepfake-irrelevant information $I_{os}$) as input, and then learns to produce an attention map that highlights the significance (potential) of the input data being correlated with deepfake-relevant (domain-relevant) information. "cat" means concatenating all input data along the channel dimension; $\otimes$ represents the Hadamard product; "c-" represents cross-channel; "s-" represents cross-spatial; "GAP", "GMP", and "ADP" are global average pooling, global max pooling, and adaptive pooling, respectively.
• Figure 4: Effect of different $\alpha$ values (used as the balance factor between BCE and AUC loss) on the AUC score. (a) is AUC with different $\alpha$ values, (b) is ROC with different $\alpha$ values.
• Figure 5: Confusion matrix visualization of domain feature classification. Each deepfake technique is recognized by the domain classification module with high accuracy (the value on the diagonal).