All in One: Unifying Deepfake Detection, Tampering Localization, and Source Tracing with a Robust Landmark-Identity Watermark

TL;DR

Extensive experiments show that the proposed LIDMark framework provides a unified, robust, and imperceptible solution for the detection, localization, and tracing of deepfake content.

Abstract

With the rapid advancement of deepfake technology, malicious face manipulations pose a significant threat to personal privacy and social security. However, existing proactive forensics methods typically treat deepfake detection, tampering localization, and source tracing as independent tasks, lacking a unified framework to address them jointly. To bridge this gap, we propose a unified proactive forensics framework that jointly addresses these three core tasks. Our core framework adopts an innovative 152-dimensional landmark-identity watermark termed LIDMark, which structurally interweaves facial landmarks with a unique source identifier. To robustly extract the LIDMark, we design a novel Factorized-Head Decoder (FHD). Its architecture factorizes the shared backbone features into two specialized heads (i.e., regression and classification), robustly reconstructing the embedded landmarks and identifier, respectively, even when subjected to severe distortion or tampering. This design realizes an "all-in-one" trifunctional forensic solution: the regression head underlies an "intrinsic-extrinsic" consistency check for detection and localization, while the classification head robustly decodes the source identifier for tracing. Extensive experiments show that the proposed LIDMark framework provides a unified, robust, and imperceptible solution for the detection, localization, and tracing of deepfake content. The code is available at https://github.com/vpsg-research/LIDMark.

Figures (6)

• Figure 1: Comparison of proactive deepfake forensic paradigms. Conventional approaches shown at the top are limited to single tasks or require complex dual-decoder architectures for bifunctional forensics. The proposed "all-in-one" framework illustrated at the bottom employs the trifunctional LIDMark and a novel FHD for deepfake detection, source tracing, and tampering localization.
• Figure 2: The 152-D LIDMark $W$ construction. The composite watermark concatenates two primary streams: (1) $W_L$, a 136-D vector of normalized 2-D facial landmarks, and (2) $W_{ID}$, a 16-D bipolar identifier derived via a SHA-256 hash of the filename.
• Figure 3: Overview of the LIDMark framework. The trifunctional forensic framework features an encoder $E$, a stochastic manipulation operator $\mathcal{M}$, a factorized-head decoder FHD, and a discriminator $D$. (a) The encoder embeds the LIDMark $W$ into $I_{co}$ via a two-stream fusion network, yielding the watermarked image $I_{wm}$. (b) $\mathcal{M}$ simulates diverse common distortions and deepfake manipulations on $I_{wm}$, producing the manipulated image $I'_{wm}$. (c) The FHD recovers $\hat{W}_L$ and $\hat{W}_{ID}$ from $I'_{wm}$ via the shared backbone and factorized heads. (d) The multi-task loss functions guide the training process. (e) The "intrinsic-extrinsic" consistency check employs recovered landmarks $\hat{W}_L$ and re-detected landmarks $W_{new}$ for fine-grained tamper detection and localization, while identifiers $\hat{W}_{ID}$ are extracted for source tracing.
• Figure 4: Visual assessment of LIDMark robustness and imperceptibility. Comparing rows 1 and 2 shows the watermarked image $I_{wm}$ is indistinguishable from the cover image $I_{co}$. Row 3 displays the manipulation results $\mathcal{M}(I_{wm})$. The "intrinsic-extrinsic" consistency check compares the green dots in row 4, representing FHD-recovered intrinsic landmarks $\hat{W}_L$, against the red dots in row 5, representing re-detected extrinsic landmarks $W_{new}$. Row 6 merges these two landmark sets into a combined image to visualize their spatial differences.
• Figure 5: Landmarks Average Euclidean Distance (AED) under various attacks. AED measures the geometric error between FHD-recovered landmarks $\hat{W}_L$ and re-detected landmarks $W_{new}$ on $\mathcal{M}(I_{wm})$. The distribution shows clear separability between benign operations and malicious attacks. The blue $\mathcal{M}_{c}$ bars yield low AED, while the red $\mathcal{M}_{d}$ bars introduce large geometric change, enabling the "intrinsic-extrinsic" consistency check.