Harnessing Frequency Spectrum Insights for Image Copyright Protection Against Diffusion Models

TL;DR

The paper reveals that diffusion-generated images retain the spectral properties of their training data, a property leveraged to protect image copyrights via CoprGuard, a spectrum-based watermarking framework. By embedding watermarks in the DWT domain with a HiNet-based encoder and counteracting potential erosion with an Information Enhancement Module, CoprGuard enables black-box detection of unauthorized image usage during diffusion-model training or fine-tuning. Empirical results across DDIM, CF-Guidance, and Stable Diffusion on FFHQ, ImageNet, and Pokemon demonstrate 100% infringement-detection accuracy even when watermarked data comprise as little as 1% of the training set, with minimal impact on image quality and strong robustness to transformations. The work provides a practical, model-agnostic approach to safeguard IP in AI-driven image generation and opens directions for extending spectral-watermarking techniques and leakage analysis in diffusion-based systems.

Abstract

Diffusion models have achieved remarkable success in novel view synthesis, but their reliance on large, diverse, and often untraceable Web datasets has raised pressing concerns about image copyright protection. Current methods fall short in reliably identifying unauthorized image use, as they struggle to generalize across varied generation tasks and fail when the training dataset includes images from multiple sources with few identifiable (watermarked or poisoned) samples. In this paper, we present novel evidence that diffusion-generated images faithfully preserve the statistical properties of their training data, particularly reflected in their spectral features. Leveraging this insight, we introduce \emph{CoprGuard}, a robust frequency domain watermarking framework to safeguard against unauthorized image usage in diffusion model training and fine-tuning. CoprGuard demonstrates remarkable effectiveness against a wide range of models, from naive diffusion models to sophisticated text-to-image models, and is robust even when watermarked images comprise a mere 1\% of the training dataset. This robust and versatile approach empowers content owners to protect their intellectual property in the era of AI-driven image generation.

Figures (11)

• Figure 1: Our work reveals a key property of diffusion models: generated images tend to retain the statistical properties of their training data---a finding that inspired our image copyright protection approach for diffusion models. This figure highlights the close similarity in Discrete Fourier Transform (DFT) spectra between original training images (top row) and diffusion-generated images (second row) across three diffusion models: DDPM ddpm, DDIM ddim, and Classifier-Free Guidance Classifier-Free. The color scale is set to [$10^{-6}$, $10^{-1}$], and the bottom row illustrates the spectral residuals and the cosine similarities (COS) chowdhury2010introduction.
• Figure 2: Mean DWT diagonal component (cD) of training images (1st row) and generated images (2nd row) for GAN and diffusion models.
• Figure 3: The RAPSD of the image with varying diffusion steps, revealing that the diffusion process progressively degrades high-frequency content and converts the image into Gaussian noise with a uniform spectral density.
• Figure 4: CoprGuard Fraemwork. The watermark image is embedded into DWT components of images using the pretrained watermark encoder, which consists of wavelet transform block and invertible neural network (INN). These images are involved into the training or finetuning of naive and text-to-image diffusion models. During inference, an image, sampled from the inspected model, is fed into the Information Enhancement Module (IEM) and pretrained watermark extractor to retrieve the watermark image $W'$. If the cosine similarity ($COS$) between the watermark $W'$ and genuine watermark $W$ exceeds the threshold $i$, the training dataset contains watermarked images.
• Figure 5: The customized HiNet significantly improves watermark extraction for AutoencoderKL-processed images. (a)-(c) show the clean image, watermarked image and watermark. (d) shows the extracted watermark for clean image. (e) and (f) show the extracted watermarks for AutoencoderKL-processed images by naive and customized HiNet, respectively.