Imperceptible Protection against Style Imitation from Diffusion Models

TL;DR

Impasto tackles the problem of protecting artworks from style imitation by diffusion models without sacrificing image fidelity. It introduces perception-aware protection, difficulty-aware protection, and a perceptual constraint bank to adapt perturbations across the image and perceptual spaces, enabling robust protection when integrated with existing methods. Through extensive experiments on painting and cartoon datasets, along with ablations and user studies, it demonstrates improved imperceptibility and comparable protection efficacy across diverse baselines and model variants, including black-box and personalization pipelines. The work highlights practical implications for copyright protection in AI-generated art and points to future work on speeding optimization and extending to broader domains.

Abstract

Recent progress in diffusion models has profoundly enhanced the fidelity of image generation, but it has raised concerns about copyright infringements. While prior methods have introduced adversarial perturbations to prevent style imitation, most are accompanied by the degradation of artworks' visual quality. Recognizing the importance of maintaining this, we introduce a visually improved protection method while preserving its protection capability. To this end, we devise a perceptual map to highlight areas sensitive to human eyes, guided by instance-aware refinement, which refines the protection intensity accordingly. We also introduce a difficulty-aware protection by predicting how difficult the artwork is to protect and dynamically adjusting the intensity based on this. Lastly, we integrate a perceptual constraints bank to further improve the imperceptibility. Results show that our method substantially elevates the quality of the protected image without compromising on protection efficacy.

Figures (17)

• Figure 1: Model overview.(a) Current protection methods update perturbation $\delta$ with an iterative process. (b) On this basis, Impasto constructs a perceptual map $\mathcal{M}$ and a difficulty map $\mathcal{M_D}$ to adaptively refine the perturbation for imperceptibility. Impasto also employs a constraint bank in the process to achieve better imperceptibility. Red-bordered boxes indicate trainable parameters. IWR denotes instance-wise refinement.
• Figure 2: Comparison of restriction types. (a) Original image. The red box shows the region with applied partial protection. (b–d) We compare partial and full protection by fine-tuning diffusion models on protected (or original) images with DreamBooth ruiz2023dreambooth. In the partial protection setting, perturbations are applied only to the central (or facial) area. The results show that partial protection is insufficient for style preservation, as personalization methods can still exploit unprotected regions to capture the artwork’s style.
• Figure 3: Examples of perceptual maps. In this study, we use luminance adaptation (LA), contrast masking (CM), contrast sensitivity function (CSF), standard deviation (Stdev), and entropy. (Left) Perceptual Map visualization. Darker region corresponds to increased protection intensity. Impasto constructs perceptual map $\mathcal{M}$ by spatially averaging these estimations (Avgerage) or an learnable manner (IWR). (Right) Perceptual map comparison. We evaluate the protection performance of each perceptual map. Image quality is assessed through DISTS ding2020image, and protection performance is evaluated via FID heusel2017gans. The baseline is trained with Eq. \ref{['eq:pgd']} and other models are trained via Eq. \ref{['eq:pap']} with corresponding JNDs.
• Figure 4: Protection difficulty. We present the protection efficacy of PhotoGuard salman2023raising across 20 artworks (x-axis). Notably, the level of style protection varies widely between artworks, thus the average score (red dashed line) is less meaningful to users who protect their artwork.
• Figure 5: Perturbation amplification. Starting from protected images (left), the diffusion process (middle) amplifies imperceptible perturbations into visible distortions. These artifacts closely resemble those in the personalized outputs (right), suggesting that the diffusion trajectory exposes directions that personalization can exploit.