Fantastic Copyrighted Beasts and How (Not) to Generate Them

TL;DR

This work tackles copyright infringement risks in image and video generation, focusing on copyrighted characters. It introduces a dual-maceted evaluation framework (DETECT and CONS) and a systematic method to identify indirect anchors that can trigger character generation without explicit names. Through experiments across multiple models, it shows that indirect anchors are effective and that prompt rewriting alone is insufficient, advocating a combined approach with negative prompts to better protect rights. The study provides actionable guidance for model deployers and highlights avenues for improving runtime safeguards and evaluation protocols in copyright-sensitive contexts.

Abstract

Recent studies show that image and video generation models can be prompted to reproduce copyrighted content from their training data, raising serious legal concerns about copyright infringement. Copyrighted characters (e.g., Mario, Batman) present a significant challenge: at least one lawsuit has already awarded damages based on the generation of such characters. Consequently, commercial services like DALL-E have started deploying interventions. However, little research has systematically examined these problems: (1) Can users easily prompt models to generate copyrighted characters, even if it is unintentional?; (2) How effective are the existing mitigation strategies? To address these questions, we introduce a novel evaluation framework with metrics that assess both the generated image's similarity to copyrighted characters and its consistency with user intent, grounded in a set of popular copyrighted characters from diverse studios and regions. We show that state-of-the-art image and video generation models can still generate characters even if characters' names are not explicitly mentioned, sometimes with only two generic keywords (e.g., prompting with "videogame, plumber" consistently generates Nintendo's Mario character). We also introduce semi-automatic techniques to identify such keywords or descriptions that trigger character generation. Using this framework, we evaluate mitigation strategies, including prompt rewriting and new approaches we propose. Our findings reveal that common methods, such as DALL-E's prompt rewriting, are insufficient alone and require supplementary strategies like negative prompting. Our work provides empirical grounding for discussions on copyright mitigation strategies and offers actionable insights for model deployers implementing these safeguards.

Figures (16)

• Figure 1: Examples of copyrighted characters generated by the open-source Playground v2.5 model li2024playground and proprietary DALL·E 3 model. For Mario (a) and Batman (b), both models can generate these characters through indirect anchoring, using relevant descriptive keywords instead of character names. DALL·E 3 blocks explicit name prompts requests (character name anchoring) with content policy messages.
• Figure 2: Selection of generated images by Playground v2.5 that GPT-4V detects as the requested characters. As shown, the model is able to generate images that look highly similar to the required character with (a) or w/o the character's name in the prompt (b, c).
• Figure 3: Number of characters detected using different top keywords ranked by various methods on (a) image generation and (b) video generation models. Ranking keywords based on their co-occurrence with the character's name in the LAION corpus is the most effective and could generate more characters than using a 60-word description when only 20 keywords are used.
• Figure 4: Example of copyrighted characters generated using (a) 60-word description with DALL·E 3, and (b) five keywords from LAION with the VideoFusion VideoFusion. The video generation model also generates watermarks in its output.
• Figure 5: Images generated with Playground v2.5 using various prompt and negative prompt configurations. Prompt rewriting, combined with negative prompting, effectively reduces the likelihood of generating images that resemble copyrighted characters while ensuring the generated subjects align with the user's intent (i.e., the main characteristics are preserved), as shown in (d).