On Copyright Risks of Text-to-Image Diffusion Models

TL;DR

The paper addresses copyright risks in text-to-image diffusion models by showing that even generic prompts can trigger copyrighted content through model memorization and unstable guidance. It introduces a two-stage data generation pipeline to produce non-sensitive yet adversarial prompts, along with an attention-map–based copyright tester to detect substantial similarities between generated content and copyrighted material. A dataset of potentially copyrighted topics and annotated target images is assembled, and experiments across multiple Stable Diffusion models (including SD XL) demonstrate a high incidence of copyright-infringing outputs. The work provides a practical toolkit for copyright testing of diffusion models and highlights the urgent need for safeguards to prevent copyright violations in commercial and research settings.

Abstract

Diffusion models excel in many generative modeling tasks, notably in creating images from text prompts, a task referred to as text-to-image (T2I) generation. Despite the ability to generate high-quality images, these models often replicate elements from their training data, leading to increasing copyright concerns in real applications in recent years. In response to this raising concern about copyright infringement, recent studies have studied the copyright behavior of diffusion models when using direct, copyrighted prompts. Our research extends this by examining subtler forms of infringement, where even indirect prompts can trigger copyright issues. Specifically, we introduce a data generation pipeline to systematically produce data for studying copyright in diffusion models. Our pipeline enables us to investigate copyright infringement in a more practical setting, involving replicating visual features rather than entire works using seemingly irrelevant prompts for T2I generation. We generate data using our proposed pipeline to test various diffusion models, including the latest Stable Diffusion XL. Our findings reveal a widespread tendency that these models tend to produce copyright-infringing content, highlighting a significant challenge in this field.

Figures (14)

• Figure 1: Generate copyrighted content in ChatGPT. ChatGPT refuses to generate images when directly prompted for copyrighted material. However, adversarial prompts generated with our method that do not directly ask for copyrighted material still manage to generate copyrighted material, in this case, the Superman logo.
• Figure 2: Unstable behavior of diffusion models. Example of prompts that trigger the generation of "Great Wave off Kanagawa" and "Superman", even when prompts have semantically different meanings from the reference topic.
• Figure 3: Attention map visualization. Image shows the generation result of Stable Diffusion 2 using the prompt "the legend of zelda". Heatmaps are averaged attention maps of each text token denoted above the heatmaps. Notably, the attention map associated with the word "zelda" shows concentration on the character, indicating its significance as a pivotal keyword in generating the intended topic.
• Figure 4: Overview of prompt pruning. Fixing the hidden input for cross-attention modules, the prompt embedding and target topic embedding are passed separately as the input of cross-attention modules. We measure the output difference to check if the prompt has the same causal effect as the target topic for cross-attention modules. Prompts with small distance values are preserved after pruning.
• Figure 5: Illustration of the copyright test. (a): Generated image. (b): Attention map of the generated image. (c): Corresponding region of interest extracted by masking with the attention map. (d): Target image and bounding box annotation. Copyright test works by finding regions similar to the annotated region in target images. The red bounding box in (c) shows the identification result.

Theorems & Definitions (2)

• Definition 1
• Definition 2