Protective Perturbations against Unauthorized Data Usage in Diffusion-based Image Generation

TL;DR

The paper tackles privacy/IP concerns from unauthorized data usage in diffusion-based image generation by surveying protective perturbation methods that render customization samples less learnable. It formalizes a defender/attacker threat model and classifies downstream tasks into text-driven synthesis (object-driven, style mimicry) and text-driven manipulation (image editing). A four-axis evaluation framework—perturbation visibility, effectiveness, cost, and robustness—standardizes assessment across methods. By consolidating existing approaches and proposing robust evaluation criteria, the work aims to guide future research toward more reliable protective strategies against unauthorized diffusion-based customization.

Abstract

Diffusion-based text-to-image models have shown immense potential for various image-related tasks. However, despite their prominence and popularity, customizing these models using unauthorized data also brings serious privacy and intellectual property issues. Existing methods introduce protective perturbations based on adversarial attacks, which are applied to the customization samples. In this systematization of knowledge, we present a comprehensive survey of protective perturbation methods designed to prevent unauthorized data usage in diffusion-based image generation. We establish the threat model and categorize the downstream tasks relevant to these methods, providing a detailed analysis of their designs. We also propose a completed evaluation framework for these perturbation techniques, aiming to advance research in this field.