Robust Identity Perceptual Watermark Against Deepfake Face Swapping

TL;DR

This work tackles privacy risks posed by Deepfake face swapping by introducing a robust identity perceptual watermarking framework. It encodes semantically meaningful watermarks tied to image identity, protected by a chaotic encryption scheme, and jointly trains an encoder-decoder to enable detection and source tracing without ground-truth watermarks. Across GAN- and diffusion-based swaps and in cross-dataset settings, the method achieves state-of-the-art watermark recovery, high Deepfake-detection AUC, and strong source-tracing accuracies, validating practical applicability. A noted limitation is its current focus on identity-preserving swaps, with future work aimed at extending protection to face re-enactment manipulations.

Abstract

Notwithstanding offering convenience and entertainment to society, Deepfake face swapping has caused critical privacy issues with the rapid development of deep generative models. Due to imperceptible artifacts in high-quality synthetic images, passive detection models against face swapping in recent years usually suffer performance damping regarding the generalizability issue in cross-domain scenarios. Therefore, several studies have been attempted to proactively protect the original images against malicious manipulations by inserting invisible signals in advance. However, existing proactive defense approaches demonstrate unsatisfactory results with respect to visual quality, detection accuracy, and source tracing ability. In this study, to fulfill the research gap, we propose a robust identity perceptual watermarking framework that concurrently performs detection and source tracing against Deepfake face swapping proactively. We innovatively assign identity semantics regarding the image contents to the watermarks and devise an unpredictable and nonreversible chaotic encryption system to ensure watermark confidentiality. The watermarks are robustly encoded and recovered by jointly training an encoder-decoder framework along with adversarial image manipulations. For a suspect image, falsification is accomplished by justifying the consistency between the content-matched identity perceptual watermark and the recovered robust watermark, without requiring the ground-truth. Moreover, source tracing can be accomplished based on the identity semantics that the recovered watermark carries. Extensive experiments demonstrate state-of-the-art detection and source tracing performance against Deepfake face swapping with promising watermark robustness for both cross-dataset and cross-manipulation settings.

Figures (5)

• Figure 1: Framework of the proposed approach. An original clean image $I$ is first passed to generator $G_m$ to generate the corresponding identity perceptual watermark $m$. Then, $I$ and $m$ are fed to the encoder for watermark embedding, which derives $I_\textrm{rec}$. Manipulation pools $P_\textrm{common}$ and $P_\textrm{swap}$ containing common and Deepfake manipulations help enhance watermark robustness during supervision. Besides, a discriminator helps maintain the visual quality. In the end, the watermark $m_\textrm{rec}$ is recovered by passing the manipulated image to the decoder.
• Figure 2: Visual effects of common image manipulations on the watermarked images. The first two columns refer to the raw and watermarked images, and each of the rest rows displays the visual effects of a manipulation algorithm on the watermarked images.
• Figure 3: Visual effects of Deepfake face swapping image manipulations on clean and watermarked images. The first column refers to the source images that provide facial identity for face swapping. Columns 2 and 3 present the raw and watermarked target images. Every two columns of the remaining exhibit the face swapping visualizations on the raw and watermarked target images.
• Figure 4: Bifurcation diagram of the logistic map Eqn.(\ref{['eq:logistic_map']}) for continuous $r$ values with $x_0 = 0.1$.
• Figure 5: Bifurcation diagrams with different $x_0$ values.