RecoverMark: Robust Watermarking for Localization and Recovery of Manipulated Faces

TL;DR

RecoverMark is a watermarking framework that achieves robust manipulation localization, content recovery, and ownership verification simultaneously and exploits a critical real-world constraint: an adversary must preserve the background's semantic consistency to avoid visual detection, even if they apply global, imperceptible watermark removal attacks.

Abstract

The proliferation of AI-generated content has facilitated sophisticated face manipulation, severely undermining visual integrity and posing unprecedented challenges to intellectual property. In response, a common proactive defense leverages fragile watermarks to detect, localize, or even recover manipulated regions. However, these methods always assume an adversary unaware of the embedded watermark, overlooking their inherent vulnerability to watermark removal attacks. Furthermore, this fragility is exacerbated in the commonly used dual-watermark strategy that adds a robust watermark for image ownership verification, where mutual interference and limited embedding capacity reduce the fragile watermark's effectiveness. To address the gap, we propose RecoverMark, a watermarking framework that achieves robust manipulation localization, content recovery, and ownership verification simultaneously. Our key insight is twofold. First, we exploit a critical real-world constraint: an adversary must preserve the background's semantic consistency to avoid visual detection, even if they apply global, imperceptible watermark removal attacks. Second, using the image's own content (face, in this paper) as the watermark enhances extraction robustness. Based on these insights, RecoverMark treats the protected face content itself as the watermark and embeds it into the surrounding background. By designing a robust two-stage training paradigm with carefully crafted distortion layers that simulate comprehensive potential attacks and a progressive training strategy, RecoverMark achieves a robust watermark embedding in no fragile manner for image manipulation localization, recovery, and image IP protection simultaneously. Extensive experiments demonstrate the proposed RecoverMark's robustness against both seen and unseen attacks and its generalizability to in-distribution and out-of-distribution data.

Figures (8)

• Figure 1: Visualization of face manipulation localization for Imuge+ ProactiveImmunePlus, EditGuard BaselineEditGuard, OmniGuard BaselineOmniGuard, and our proposed RecoverMark. The evaluation is conducted using three distinct region-of-interest definitions: GSAM2 gsam (face-only segmentation), MTCNN mtcnn (head bounding box), and YOLOSeg yoloseg (full-body). All methods are tested against traditional low-pass filtering and an advanced regeneration attack attack_regeneration, where $I_{\text{ori}}$ denotes the original image, $I_{\text{sal}}$ represents segmented saliency (face), $I_{\text{bg}}$ refers to segmented background, and $I_{\text{tamp}}$ stands for the tampered image.
• Figure 2: A practical example when a judge determines whether the evidence is credible based on background consistency.
• Figure 3: Flowchart of the training process for the proposed RecoverMark, where MTCNN mtcnn is used as segmentation model, and the watermark encoder/decoder are trained in Stage $1$ but remain frozen during Stage $2$.
• Figure 4: Demonstration of manipulation localization, recovery, and ownership verification for a suspicious image.
• Figure 5: Demonstration of RecoverMark's robustness against a range of attacks (from single to multiple), including the unseen lattice attack Liu2023Erase_Removal_Attack, arranged from top to bottom and left to right..