ArtistAuditor: Auditing Artist Style Pirate in Text-to-Image Generation Models

TL;DR

This work tackles copyright protection for artist styles in diffusion-based text-to-image generation by proposing ArtistAuditor, a data-use auditing framework that leverages multi-granularity CNN latent representations as intrinsic fingerprints of an artist's style. A discriminator is trained on public artworks, mimicked outputs, and augmented samples to produce confidence scores indicating whether a suspicious model has been finetuned on the target artist's work, with thresholding or one-sided t-testing to make decisions. The method demonstrates high discriminative performance (AUC > 0.937) across multiple models and datasets, robustness to some variations in captioning and data overlap, and practical viability demonstrated on an online platform, while maintaining data fidelity without modifying original artworks or retraining hidden models. The work provides actionable guidance on dataset preparation, augmentation, distortion calibration, and transferability, offering a scalable, black-box auditing solution for real-world art-style copyright protection in AI-generated content.

Abstract

Text-to-image models based on diffusion processes, such as DALL-E, Stable Diffusion, and Midjourney, are capable of transforming texts into detailed images and have widespread applications in art and design. As such, amateur users can easily imitate professional-level paintings by collecting an artist's work and fine-tuning the model, leading to concerns about artworks' copyright infringement. To tackle these issues, previous studies either add visually imperceptible perturbation to the artwork to change its underlying styles (perturbation-based methods) or embed post-training detectable watermarks in the artwork (watermark-based methods). However, when the artwork or the model has been published online, i.e., modification to the original artwork or model retraining is not feasible, these strategies might not be viable. To this end, we propose a novel method for data-use auditing in the text-to-image generation model. The general idea of ArtistAuditor is to identify if a suspicious model has been finetuned using the artworks of specific artists by analyzing the features related to the style. Concretely, ArtistAuditor employs a style extractor to obtain the multi-granularity style representations and treats artworks as samplings of an artist's style. Then, ArtistAuditor queries a trained discriminator to gain the auditing decisions. The experimental results on six combinations of models and datasets show that ArtistAuditor can achieve high AUC values (> 0.937). By studying ArtistAuditor's transferability and core modules, we provide valuable insights into the practical implementation. Finally, we demonstrate the effectiveness of ArtistAuditor in real-world cases by an online platform Scenario. ArtistAuditor is open-sourced at https://github.com/Jozenn/ArtistAuditor.

Figures (8)

• Figure 1: Intuitive explanation of $\mathsf{ArtistAuditor}$. Images with orange borders represent artist A's artworks, red borders indicate artworks mimicked by models, and blue borders show B's artworks. The discriminator identifies the style pirate based on the latent representations of the artworks.
• Figure 2: An example of stylistic imitation by Stable Diffusion. Left: original artwork. Right: generated artwork.
• Figure 3: An example of the application scenario. The auditor acquires the auditing results by comparing the style representations between the original artwork of artist A and the artworks generated by the fine-tuned model.
• Figure 4: The workflow of $\mathsf{ArtistAuditor}$ contains three steps, i.e., dataset preparation, discriminator construction, and auditing process. $\mathsf{ArtistAuditor}$ first collects the public artworks and generated artworks by the suspicious model, then extracts the multi-granularity style representations to train the discriminator. Finally, $\mathsf{ArtistAuditor}$ extracts the style features of mimicked artworks and makes the auditing decisions based on the outputs of the discriminator.
• Figure 5: CLIP and BLIP generate captions for the same set of artworks, respectively.