FractalForensics: Proactive Deepfake Detection and Localization via Fractal Watermarks

TL;DR

FractalForensics addresses the challenge of proactive Deepfake detection with localization by introducing fractal watermarks generated from a parameter-driven pipeline and encrypted via chaotic maps. The method embeds a 4-bit, 4-channel watermark w^c using an entry-to-patch strategy, enabling spatially aware localization while preserving image quality through an end-to-end trainable framework. Experiments on CelebA-HQ and LFW show strong visual fidelity, high patch- and bit-wise watermark recovery, and superior AUC-based Deepfake detection compared with state-of-the-art semi-fragile and passive detectors, with explicit localization of manipulated regions. The work demonstrates the feasibility and practicality of fractal-based, semi-fragile watermarks for proactive defense and forensic explainability in real-world Deepfake scenarios, and suggests avenues for automatic fractal self-checks and extended tracing capabilities.

Abstract

Proactive Deepfake detection via robust watermarks has seen interest ever since passive Deepfake detectors encountered challenges in identifying high-quality synthetic images. However, while demonstrating reasonable detection performance, they lack localization functionality and explainability in detection results. Additionally, the unstable robustness of watermarks can significantly affect the detection performance. In this study, we propose novel fractal watermarks for proactive Deepfake detection and localization, namely FractalForensics. Benefiting from the characteristics of fractals, we devise a parameter-driven watermark generation pipeline that derives fractal-based watermarks and performs one-way encryption of the selected parameters. Subsequently, we propose a semi-fragile watermarking framework for watermark embedding and recovery, trained to be robust against benign image processing operations and fragile when facing Deepfake manipulations in a black-box setting. Moreover, we introduce an entry-to-patch strategy that implicitly embeds the watermark matrix entries into image patches at corresponding positions, achieving localization of Deepfake manipulations. Extensive experiments demonstrate satisfactory robustness and fragility of our approach against common image processing operations and Deepfake manipulations, outperforming state-of-the-art semi-fragile watermarking algorithms and passive detectors for Deepfake detection. Furthermore, by highlighting the areas manipulated, our method provides explainability for the proactive Deepfake detection results.

Figures (15)

• Figure 1: Pipeline of watermark generation and encryption based on user-chosen parameters. Given a standard Hilbert curve, parameters $r$, $m$, and $o$ determine rotation, mirroring, and order modification variations when deriving the raw watermark, while $x_0$, $a$, $k$, and $d$ determine how the watermark is securely encrypted in the chaotic system.
• Figure 2: Workflow of the proposed FractalForensics. The channel-wise watermark matrix $w^c$ is embedded into image $I$ following image feature extraction, watermark diffusion, and watermark fusion modules. To recover the watermark from $I_\textrm{rec}$, we construct a decoder. The framework is trained end-to-end. In the testing phase, Deepfake localization is implicitly achieved.
• Figure 3: Effect of image operations on the watermarked image with localization demonstrated by the red overlay.
• Figure 4: Effect of Deepfake algorithms on the watermarked image with localization demonstrated by the red overlay.
• Figure 5: Qualitative comparison of localization performance between watermarking frameworks. The localized fake areas are highlighted in purple, white, and red by WaterLo, EditGuard, and our proposed FractalForensics, respectively.