©Plug-in Authorization for Human Content Copyright Protection in Text-to-Image Model

TL;DR

This work tackles copyright infringement concerns in text-to-image generation by proposing the ©Plug-in Authorization framework, which enables attribution and monetization through three operations: addition, extraction, and combination. It introduces Reverse LoRA for extracting copyrighted concepts into plug-ins and EasyMerge for data-free, layer-wise distillation to merge multiple plug-ins, all within a diffusion-model setting like Stable Diffusion. Empirical results on artist styles and cartoon IPs demonstrate effective target-concept removal with preserved surrounding styles and successful IP recreation when plug-ins are combined, offering a practical pathway for fair reward distribution in generative AI. The framework aims to balance creative freedom with copyright protection by embedding attribution and usage rewards directly into the model’s fine-tuning workflow, rather than relying solely on data removal or post-filtering.

Abstract

This paper addresses the contentious issue of copyright infringement in images generated by text-to-image models, sparking debates among AI developers, content creators, and legal entities. State-of-the-art models create high-quality content without crediting original creators, causing concern in the artistic community. To mitigate this, we propose the ©Plug-in Authorization framework, introducing three operations: addition, extraction, and combination. Addition involves training a ©plug-in for specific copyright, facilitating proper credit attribution. Extraction allows creators to reclaim copyright from infringing models, and combination enables users to merge different ©plug-ins. These operations act as permits, incentivizing fair use and providing flexibility in authorization. We present innovative approaches,"Reverse LoRA" for extraction and "EasyMerge" for seamless combination. Experiments in artist-style replication and cartoon IP recreation demonstrate ©plug-ins' effectiveness, offering a valuable solution for human copyright protection in the age of generative AIs. The code is available at https://github.com/zc1023/-Plug-in-Authorization.git.

Figures (12)

• Figure 1: © Plug-in Authorization Process. The authorization process consists of three types of entities: user, model provider, and IP owners (artists). Users can generate copyrighted images only by accessing the relevant plug-in. The model provider offers services to users, tracks usage of plug-ins, and attributes rewards to the IP owner. The IP owners can achieve authorization by registering their © plug-ins through addition or extraction. These © plug-ins form a pool where users can get © plug-ins to produce content with the IP owner's authorization.
• Figure 2: Three foundational operations achieving © Plug-in Authorization: addition, extraction, and combination. The plug-in can be created by addition if the copyrighted work is new to the base model. Meanwhile, the plug-in can be created by extracting from the base model if the copyrighted work is already infringed by the base model. Once a pool of © plug-ins is constructed, the combination operation can merge multiple © plug-ins featuring the generation of multiple concepts and leave a non-infringing model excluding all multiple concepts.
• Figure 3: The method of extraction consists of two steps:de-concept and re-context. The de-concept step tries to capture the target concept "Picasso" by tuning the LoRA component to match copyrighted images with the contextual prompt "The painting of building". In the re-context step, we reverse the LoRA (so that successfully forget "Picasso") and then further tune the LoRA with surrounding contextual prompt and non-copyrighted image pairs, to ensure the capabilities of contextual generation.
• Figure 4: Results of style replication. In Figure (a), we show samples from different non-infringing models in each column. Each non-infringing model exhibits a deficiency in one style generation ability, with all other style generation capabilities remaining unaffected. In Figure (b), we present samples generated after integrating certain © Plug-ins in each column. Each of these © Plug-ins serves to exclusively restore the generation of one particular style, while the generation of other styles continues to exhibit diminished performance.
• Figure 5: Results of IP Recreation. Each column on the right represents the output of a distinct non-infringing model. We successfully extract the unique IPs of Mickey, R2D2, and Vader independently, preserving the generation of other IPs.