Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Removing Watermarks with Partial Regeneration using Semantic Information

Krti Tallam, John Kevin Cava, Caleb Geniesse, N. Benjamin Erichson, Michael W. Mahoney

TL;DR

The paper addresses the vulnerability gap in semantic and invisible watermarks when faced with adaptive, content-preserving attacks. It proposes SemanticRegen, a three-stage pipeline that leverages VQA captioning (BLIP2), LangSAM-based segmentation, and LLM-guided background inpainting (MiniChat-MA + Stable Diffusion) to erase watermark signals while preserving foreground meaning, quantified by the novel mSSIM metric. Empirically, SemanticRegen defeats the state-of-the-art semantic TreeRing watermark (average $p$-value $0.10$) and reduces Bit Accuracy to $<0.75$ for StegaStamp, StableSig, and DWT/DCT across 1,000 prompts, while maintaining high perceptual fidelity (masked SSIM about $0.94$–$0.95$). These findings reveal a significant gap between current watermark defenses and what clever, semantics-aware attackers can achieve, underscoring the need for more robust watermarking schemes and standardized evaluation frameworks like WAVES, as well as governance considerations for AI-generated content protection.

Abstract

As AI-generated imagery becomes ubiquitous, invisible watermarks have emerged as a primary line of defense for copyright and provenance. The newest watermarking schemes embed semantic signals - content-aware patterns that are designed to survive common image manipulations - yet their true robustness against adaptive adversaries remains under-explored. We expose a previously unreported vulnerability and introduce SemanticRegen, a three-stage, label-free attack that erases state-of-the-art semantic and invisible watermarks while leaving an image's apparent meaning intact. Our pipeline (i) uses a vision-language model to obtain fine-grained captions, (ii) extracts foreground masks with zero-shot segmentation, and (iii) inpaints only the background via an LLM-guided diffusion model, thereby preserving salient objects and style cues. Evaluated on 1,000 prompts across four watermarking systems - TreeRing, StegaStamp, StableSig, and DWT/DCT - SemanticRegen is the only method to defeat the semantic TreeRing watermark (p = 0.10 > 0.05) and reduces bit-accuracy below 0.75 for the remaining schemes, all while maintaining high perceptual quality (masked SSIM = 0.94 +/- 0.01). We further introduce masked SSIM (mSSIM) to quantify fidelity within foreground regions, showing that our attack achieves up to 12 percent higher mSSIM than prior diffusion-based attackers. These results highlight an urgent gap between current watermark defenses and the capabilities of adaptive, semantics-aware adversaries, underscoring the need for watermarking algorithms that are resilient to content-preserving regenerative attacks.

Removing Watermarks with Partial Regeneration using Semantic Information

TL;DR

The paper addresses the vulnerability gap in semantic and invisible watermarks when faced with adaptive, content-preserving attacks. It proposes SemanticRegen, a three-stage pipeline that leverages VQA captioning (BLIP2), LangSAM-based segmentation, and LLM-guided background inpainting (MiniChat-MA + Stable Diffusion) to erase watermark signals while preserving foreground meaning, quantified by the novel mSSIM metric. Empirically, SemanticRegen defeats the state-of-the-art semantic TreeRing watermark (average -value ) and reduces Bit Accuracy to for StegaStamp, StableSig, and DWT/DCT across 1,000 prompts, while maintaining high perceptual fidelity (masked SSIM about –). These findings reveal a significant gap between current watermark defenses and what clever, semantics-aware attackers can achieve, underscoring the need for more robust watermarking schemes and standardized evaluation frameworks like WAVES, as well as governance considerations for AI-generated content protection.

Abstract

As AI-generated imagery becomes ubiquitous, invisible watermarks have emerged as a primary line of defense for copyright and provenance. The newest watermarking schemes embed semantic signals - content-aware patterns that are designed to survive common image manipulations - yet their true robustness against adaptive adversaries remains under-explored. We expose a previously unreported vulnerability and introduce SemanticRegen, a three-stage, label-free attack that erases state-of-the-art semantic and invisible watermarks while leaving an image's apparent meaning intact. Our pipeline (i) uses a vision-language model to obtain fine-grained captions, (ii) extracts foreground masks with zero-shot segmentation, and (iii) inpaints only the background via an LLM-guided diffusion model, thereby preserving salient objects and style cues. Evaluated on 1,000 prompts across four watermarking systems - TreeRing, StegaStamp, StableSig, and DWT/DCT - SemanticRegen is the only method to defeat the semantic TreeRing watermark (p = 0.10 > 0.05) and reduces bit-accuracy below 0.75 for the remaining schemes, all while maintaining high perceptual quality (masked SSIM = 0.94 +/- 0.01). We further introduce masked SSIM (mSSIM) to quantify fidelity within foreground regions, showing that our attack achieves up to 12 percent higher mSSIM than prior diffusion-based attackers. These results highlight an urgent gap between current watermark defenses and the capabilities of adaptive, semantics-aware adversaries, underscoring the need for watermarking algorithms that are resilient to content-preserving regenerative attacks.
Paper Structure (28 sections, 1 equation, 4 figures, 3 tables)

This paper contains 28 sections, 1 equation, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Watermarking Methods.
  4. Watermark Removal Methods.
  5. Ethical and practical implications.
  6. Limitations and open challenges.
  7. Broader Context.
  8. Methods
  9. VQA Captioning
  10. Structured Prompting for Semantic Understanding.
  11. VQA-Guided Watermark Removal.
  12. Implementation Details.
  13. Key Assumptions.
  14. Segmentation Model
  15. Integration with Visual Question Answering (VQA).
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Our semantic watermark removal pipeline involves three primary components: (1) Captioning (green), (2) Segmentation (red), and (3) Inpainting (blue). For captioning, we use a VQA model to provide essential context for subsequent processing. For segmentation, we focus on prominent objects or areas of interest within the image. For inpainting, the background of the image is replaced with semantically similar content, effectively removing the watermark while preserving image integrity. To construct the prompt for conditional text inpainting, we use MiniChat-MA, an LLM that refines answers generated from the image captioning model. This pipeline extracts semantic information and replaces the background for watermark removal, while preserving the foreground content.
  • Figure 2: Examples before and after watermarking with Tree Ring, and SemanticRegen. Segmentation masks used during the attack are shown in the bottom row.
  • Figure 3: Comparison of images displaying different watermarks before and after undergoing our attack methods. SemanticRegen produces significantly higher quality images compared to Image Distortion and Rinse4x. For detailed metrics, see Table \ref{['table:tab1']}.
  • Figure 4: Performance versus image quality comparison. Points further to the right indicate better (masked) image quality. These results demonstrate that SemanticRegen (purple) effectively preserves vital parts of the image while disrupting watermark components (contrast with other colors). This balance allows our framework to outperform other attackers in terms of image quality, while still maintaining its ability to disrupt watermark integrity. In contrast, other attackers (other colors) exhibit diminished image quality, even when excelling in some performance metrics. For more details, refer to Section \ref{['sec4.4']}.