EditShield: Protecting Unauthorized Image Editing by Instruction-guided Diffusion Models

TL;DR

EditShield addresses the risk of unauthorized edits by instruction-guided diffusion models by injecting imperceptible perturbations that shift the latent representation of the original image. It optimizes a prompt-agnostic objective that maximizes latent-discrepancy while enforcing perceptual consistency and transformation robustness through Expectation Over Transformation. Empirically, it reduces editor success on both synthetic and real-world datasets, outperforming prior protections and showing robustness to editing types and synonymous prompts, albeit with limitations in black-box scenarios. This work provides a practical, general approach to safeguarding visual content against flexible instruction-guided edits with implications for privacy and content integrity in shared media.

Abstract

Text-to-image diffusion models have emerged as an evolutionary for producing creative content in image synthesis. Based on the impressive generation abilities of these models, instruction-guided diffusion models can edit images with simple instructions and input images. While they empower users to obtain their desired edited images with ease, they have raised concerns about unauthorized image manipulation. Prior research has delved into the unauthorized use of personalized diffusion models; however, this problem of instruction-guided diffusion models remains largely unexplored. In this paper, we first propose a protection method EditShield against unauthorized modifications from such models. Specifically, EditShield works by adding imperceptible perturbations that can shift the latent representation used in the diffusion process, tricking models into generating unrealistic images with mismatched subjects. Our extensive experiments demonstrate EditShield's effectiveness among synthetic and real-world datasets. Besides, we found that EditShield performs robustly against various manipulation settings across editing types and synonymous instruction phrases.

Figures (13)

• Figure 1: The illustration of protection by EditShield. Editors cannot get their expected images with the protection of EditShield.
• Figure 2: The workflow of instructional image editing based on diffusion models.
• Figure 3: The image editing scenario that we consider.
• Figure 4: Qualitative protection results. Each pair of rows displays the source and protected images with their respective edited versions.
• Figure 5: Quantitative results on editing with protection of PhotoGuard and EditShield.