Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models

TL;DR

Glaze tackles the risk of AI style mimicry by diffusion models through style cloaks that impart imperceptible perturbations to artists' online artwork. By guiding perturbations with style transfer toward a dissimilar target style and constraining changes with LPIPS, Glaze disrupts the feature-space learning that underpins mimicry, without noticeably harming the original art. Large-scale artist studies and CLIP-based assessments show high protection efficacy, both under normal conditions and against adaptive countermeasures. The work demonstrates practical viability, real-world robustness, and meaningful engagement with the professional art community, offering a significant, user-centered approach to safeguarding artistic styles online.

Abstract

Recent text-to-image diffusion models such as MidJourney and Stable Diffusion threaten to displace many in the professional artist community. In particular, models can learn to mimic the artistic style of specific artists after "fine-tuning" on samples of their art. In this paper, we describe the design, implementation and evaluation of Glaze, a tool that enables artists to apply "style cloaks" to their art before sharing online. These cloaks apply barely perceptible perturbations to images, and when used as training data, mislead generative models that try to mimic a specific artist. In coordination with the professional artist community, we deploy user studies to more than 1000 artists, assessing their views of AI art, as well as the efficacy of our tool, its usability and tolerability of perturbations, and robustness across different scenarios and against adaptive countermeasures. Both surveyed artists and empirical CLIP-based scores show that even at low perturbation levels (p=0.05), Glaze is highly successful at disrupting mimicry under normal conditions (>92%) and against adaptive countermeasures (>85%).

Figures (23)

• Figure 1: Sample AI-generated art pieces from the Midjourney community showcase mid-top-artistnamewinaward.
• Figure 2: Real-world incident of AI plagiarizing the style of artist Hollie Mengert hollie-steal. Left: original artwork by Hollie Mengert. Right: plagiarized artwork generated by a model trained to mimic Hollie's style.
• Figure 3: High level model architecture of text-to-image models.
• Figure 4: High level overview of the mimicry attack scenario. The mimic scrapes copyrighted artwork from the victim artist and uses these to fine-tune a pre-trained, generic text-to-image model. The generic model is trained and open-sourced by an AI company. The mimic then uses the fine-tuned model to generate artwork in the style of the victim artist.
• Figure 5: Overview of Glaze, a system that protects victim artists from AI style mimicry by cloaking their online artwork. ( Top) An artist $V$ applies the cloaking algorithm (uses a feature extractor $\Phi$ and a target style $T$) to generate cloaked versions of $V$'s art pieces. Each cloak is a small perturbation unnoticeable to human eye. ( Bottom) A mimic scrapes the cloaked art pieces from online and uses them to fine-tune a model to mimic $V$'s style. When prompted to generate artwork in the style of $V$, mimic's model will generate artwork in the target style $T$, rather than $V$'s true style.