DeContext as Defense: Safe Image Editing in Diffusion Transformers

TL;DR

The paper addresses privacy risks from in-context image editing with diffusion-transformer models by introducing DeContext, an attention-aware defense that perturbs the input context to disrupt cross-attention pathways. By focusing perturbations on early denoising steps and influential front-to-middle transformer blocks, DeContext achieves robust context detachment while preserving image fidelity. Empirical results on Flux Kontext and Step1X-Edit show strong protection against identity transfer, with substantial drops in recognition accuracy and minimal degradation in visual quality. This approach provides a practical, model-agnostic defense against illicit personalized editing in modern DiT-based systems, offering a foundation for further robustness and transferability improvements.

Abstract

In-context diffusion models allow users to modify images with remarkable ease and realism. However, the same power raises serious privacy concerns: personal images can be easily manipulated for identity impersonation, misinformation, or other malicious uses, all without the owner's consent. While prior work has explored input perturbations to protect against misuse in personalized text-to-image generation, the robustness of modern, large-scale in-context DiT-based models remains largely unexamined. In this paper, we propose DeContext, a new method to safeguard input images from unauthorized in-context editing. Our key insight is that contextual information from the source image propagates to the output primarily through multimodal attention layers. By injecting small, targeted perturbations that weaken these cross-attention pathways, DeContext breaks this flow, effectively decouples the link between input and output. This simple defense is both efficient and robust. We further show that early denoising steps and specific transformer blocks dominate context propagation, which allows us to concentrate perturbations where they matter most. Experiments on Flux Kontext and Step1X-Edit show that DeContext consistently blocks unwanted image edits while preserving visual quality. These results highlight the effectiveness of attention-based perturbations as a powerful defense against image manipulation.

Figures (11)

• Figure 1: Overview of our protection method against malicious edits by Flux-Kontext. A clean user image (top left) can be altered for neutral, violent, sexual, or misleading edits. Our DeContext injects imperceptible perturbations into the input image, preventing identity preservation in edited results while retaining visual quality.
• Figure 2: Attack results. Standard attack (middle) only produces re-lighting artifacts, while attention intervention (right) successfully detaches the context.
• Figure 3: Overview of our DeContext pipeline. Given a prompt, timestep, noisy target, and context image, DeContext perturbs the context to suppress its attention in the diffusion model. Iterative gradient updates minimize attention activation, detaching the context from influencing generation.
• Figure 4: Time-wise gradients analysis. Gradients of context image dominate at high timesteps (early denoising).
• Figure 5: Block-wise attention analysis. Context proportion is high in early-to-mid blocks before attack and drops afterward.