LampMark: Proactive Deepfake Detection via Training-Free Landmark Perceptual Watermarks

TL;DR

A proactive Deepfake detection approach is proposed by introducing a novel training-free landmark perceptual watermark, LampMark for short, and an end-to-end watermarking framework that imperceptibly and robustly embeds and extracts watermarks concerning the images to be protected.

Abstract

Deepfake facial manipulation has garnered significant public attention due to its impacts on enhancing human experiences and posing privacy threats. Despite numerous passive algorithms that have been attempted to thwart malicious Deepfake attacks, they mostly struggle with the generalizability challenge when confronted with hyper-realistic synthetic facial images. To tackle the problem, this paper proposes a proactive Deepfake detection approach by introducing a novel training-free landmark perceptual watermark, LampMark for short. We first analyze the structure-sensitive characteristics of Deepfake manipulations and devise a secure and confidential transformation pipeline from the structural representations, i.e. facial landmarks, to binary landmark perceptual watermarks. Subsequently, we present an end-to-end watermarking framework that imperceptibly and robustly embeds and extracts watermarks concerning the images to be protected. Relying on promising watermark recovery accuracies, Deepfake detection is accomplished by assessing the consistency between the content-matched landmark perceptual watermark and the robustly recovered watermark of the suspect image. Experimental results demonstrate the superior performance of our approach in watermark recovery and Deepfake detection compared to state-of-the-art methods across in-dataset, cross-dataset, and cross-manipulation scenarios.

Figures (7)

• Figure 1: Demonstration of the structure-sensitive characteristic. Left: visualization of landmark offsets in Euclidean distances ($\rho$) between manipulated images (in blue) and the original target image (in red). Right: Landmark offset distributions of sampled common and Deepfake manipulations.
• Figure 2: Overall framework of the proposed method. The landmark perceptual watermarks are produced via pipeline $G_m$ and encoded into raw images. After benign and Deepfake manipulations, the watermarked images are passed to the decoder for watermark recovery. By comparing the landmark perceptual watermarks of the manipulated images with the recovered watermarks, Deepfake detection is accomplished.
• Figure 3: Visual effects of the manipulations on the watermarked images. The raw and watermarked images are displayed in the bottom and middle rows. Manipulated outputs via different operations on the watermarked images are placed in the top row. The left four columns present results by benign manipulations and the remaining exhibit those by Deepfake manipulations.
• Figure 4: Visualizations of the effects for each benign manipulation.
• Figure 5: Visualization of the binary key values from time steps 0 to 9 in white and dark grids with a key length of 20.