Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating and Mitigating IP Infringement in Visual Generative AI

Zhenting Wang, Chen Chen, Vikash Sehwag, Minzhou Pan, Lingjuan Lyu

TL;DR

The paper tackles IP infringement risks in diffusion-based visual generative AI, showing that protected characters can be reproduced even from prompts that do not explicitly name them. It introduces TRIM, a defense that blocks infringing prompts via LLMs, detects potential infringements with vision-language models, and suppresses infringement during diffusion through targeted guidance, all without retraining the models. The authors formalize the problem with a black-box framework, construct a lure-prompt benchmark across six high-profile characters, and demonstrate that IP infringement is widespread across open and closed models. Empirical results show that TRIM substantially reduces infringement rates with only a modest impact on language–image alignment, highlighting a practical pathway for safer deployment of visual AIGC technologies.

Abstract

The popularity of visual generative AI models like DALL-E 3, Stable Diffusion XL, Stable Video Diffusion, and Sora has been increasing. Through extensive evaluation, we discovered that the state-of-the-art visual generative models can generate content that bears a striking resemblance to characters protected by intellectual property rights held by major entertainment companies (such as Sony, Marvel, and Nintendo), which raises potential legal concerns. This happens when the input prompt contains the character's name or even just descriptive details about their characteristics. To mitigate such IP infringement problems, we also propose a defense method against it. In detail, we develop a revised generation paradigm that can identify potentially infringing generated content and prevent IP infringement by utilizing guidance techniques during the diffusion process. It has the capability to recognize generated content that may be infringing on intellectual property rights, and mitigate such infringement by employing guidance methods throughout the diffusion process without retrain or fine-tune the pretrained models. Experiments on well-known character IPs like Spider-Man, Iron Man, and Superman demonstrate the effectiveness of the proposed defense method. Our data and code can be found at https://github.com/ZhentingWang/GAI_IP_Infringement.

Evaluating and Mitigating IP Infringement in Visual Generative AI

TL;DR

The paper tackles IP infringement risks in diffusion-based visual generative AI, showing that protected characters can be reproduced even from prompts that do not explicitly name them. It introduces TRIM, a defense that blocks infringing prompts via LLMs, detects potential infringements with vision-language models, and suppresses infringement during diffusion through targeted guidance, all without retraining the models. The authors formalize the problem with a black-box framework, construct a lure-prompt benchmark across six high-profile characters, and demonstrate that IP infringement is widespread across open and closed models. Empirical results show that TRIM substantially reduces infringement rates with only a modest impact on language–image alignment, highlighting a practical pathway for safer deployment of visual AIGC technologies.

Abstract

The popularity of visual generative AI models like DALL-E 3, Stable Diffusion XL, Stable Video Diffusion, and Sora has been increasing. Through extensive evaluation, we discovered that the state-of-the-art visual generative models can generate content that bears a striking resemblance to characters protected by intellectual property rights held by major entertainment companies (such as Sony, Marvel, and Nintendo), which raises potential legal concerns. This happens when the input prompt contains the character's name or even just descriptive details about their characteristics. To mitigate such IP infringement problems, we also propose a defense method against it. In detail, we develop a revised generation paradigm that can identify potentially infringing generated content and prevent IP infringement by utilizing guidance techniques during the diffusion process. It has the capability to recognize generated content that may be infringing on intellectual property rights, and mitigate such infringement by employing guidance methods throughout the diffusion process without retrain or fine-tune the pretrained models. Experiments on well-known character IPs like Spider-Man, Iron Man, and Superman demonstrate the effectiveness of the proposed defense method. Our data and code can be found at https://github.com/ZhentingWang/GAI_IP_Infringement.
Paper Structure (20 sections, 17 figures, 6 tables, 1 algorithm)

This paper contains 20 sections, 17 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. IP Infringement Evaluation
  4. Problem Formulation
  5. Constructing Lure Prompt for IP Infringement Evaluation
  6. Experiments
  7. Mitigation Method
  8. Problem Formulation
  9. Overview of Our Mitigation Approach
  10. Exploiting the Perception and Understanding Capability of L(V)LMs
  11. Infringement Suppression
  12. Algorithm
  13. Mitigation Experiments
  14. Effectiveness
  15. Ablation Study
  16. ...and 5 more sections

Figures (17)

  • Figure 1: Generated samples of different the state-of-the-art visual generative AIs by using the prompt "Generate an image of the Spider-Man". Images are generated in April, 2024. The generated contents violate the IP of the "Spider-Man".
  • Figure 2: Generated samples of different state-of-the-art visual generative AIs by using the prompt "Imagine a superhero clad in a sleek, skin-tight suit, primarily red with distinctive blue patterns across the arms, chest, and legs. The suit has a web-like design subtly integrated throughout. This character has large, expressive eyes on the mask, designed in a white, reflective material to give a mysterious and captivating appearance. The hero is poised on top of a towering city skyscraper, crouched in a dynamic pose, ready to leap into action. The backdrop shows a bustling urban landscape at dusk, the sky tinged with hues of orange and purple. This superhero's persona is one of agility and strength, and their posture suggests they are about to use their remarkable acrobatic skills to swing between the buildings." Images are generated in April, 2024. The generated contents violate the IP of the "Spider-Man".
  • Figure 3: Prompt for Constructing Lure that can trigger IP Infringement on the target character
  • Figure 4: Visualizations of the involved characters. The source of these images are listed in \ref{['sec:appendix_source']}.
  • Figure 5: Generated samples of different the state-of-the-art visual generative AIs by using the prompt "Generate an image of the Iron Man." Images are generated in April, 2024. The generated contents violate the IP of the "Iron Man".
  • ...and 12 more figures

Theorems & Definitions (1)

  • Definition 1