Capture Artifacts via Progressive Disentangling and Purifying Blended Identities for Deepfake Detection

TL;DR

A Deepfake detection method based on progressive disentangling and purifying blended identities is innovatively proposed in this paper, which aims to more accurately capture and separate artifact features in fake faces.

Abstract

The Deepfake technology has raised serious concerns regarding privacy breaches and trust issues. To tackle these challenges, Deepfake detection technology has emerged. Current methods over-rely on the global feature space, which contains redundant information independent of the artifacts. As a result, existing Deepfake detection techniques suffer performance degradation when encountering unknown datasets. To reduce information redundancy, the current methods use disentanglement techniques to roughly separate the fake faces into artifacts and content information. However, these methods lack a solid disentanglement foundation and cannot guarantee the reliability of their disentangling process. To address these issues, a Deepfake detection method based on progressive disentangling and purifying blended identities is innovatively proposed in this paper. Based on the artifact generation mechanism, the coarse- and fine-grained strategies are combined to ensure the reliability of the disentanglement method. Our method aims to more accurately capture and separate artifact features in fake faces. Specifically, we first perform the coarse-grained disentangling on fake faces to obtain a pair of blended identities that require no additional annotation to distinguish between source face and target face. Then, the artifact features from each identity are separated to achieve fine-grained disentanglement. To obtain pure identity information and artifacts, an Identity-Artifact Correlation Compression module (IACC) is designed based on the information bottleneck theory, effectively reducing the potential correlation between identity information and artifacts. Additionally, an Identity-Artifact Separation Contrast Loss is designed to enhance the independence of artifact features post-disentangling. Finally, the classifier only focuses on pure artifact features to achieve a generalized Deepfake detector.

Figures (5)

• Figure 1: (a) The flowchart of the proposed Deepfake detection method based on progressive disentanglement; (b) Venn diagram illustrating the identity-artifact relationship, supporting the rationale and process of the disentanglement approach; (c) The effect of the Identity-Artifact Separation Contrastive loss: pushing the pure artifact features away from the identity of the fake face, while pulling the identity of the real face closer to the pseudo artifact features.
• Figure 2: Framework diagram of the Deepfake detection method based on progressive disentangling and purifying blended identities. The network input is a pair of images, consisting of a real face image and a fake face image. The main components of the framework are as follows: 1) Identity Extraction (Coarse-Grained). 2) Artifact Separation (Fine-Grained). 3) Correlation Compression and Aggregation. 4) Face self-reconstruction and cross-reconstruction. 5) Deepfake Detection.
• Figure 3: The t-SNE visualization illustrates the feature distributions of identity information and artifacts. (a) t-SNE visualization of unprocessed mixed identity information. (b) t-SNE visualization of identity information and artifacts after applying the basic progressive disentanglement framework (w/o IACC+$L_{Con}$). (c) t-SNE visualization of the pure identity information and pure artifacts obtained using our proposed method.
• Figure 4: The t-SNE visualization of the multi-domain feature distribution extracted from the FF++ (c23) dataset by EfficientNet-B4 and our proposed method.
• Figure 5: Visualization of face self-reconstruction and cross-reconstruction in the progressive disentanglement framework.