SpotEdit: Selective Region Editing in Diffusion Transformers

TL;DR

SpotEdit addresses the inefficiency of full-image regeneration in diffusion-transformer editing by introducing a training-free, region-aware approach. It combines SpotSelector, which uses a perceptual LPIPS-like score to identify non-edited regions, with SpotFusion, which temporally blends non-edited features into the edited region via a cosine-squared schedule and partial-attention, ensuring coherence across timesteps. Empirical results show substantial speedups (around 1.7–1.95×) on standard benchmarks while maintaining or improving editing fidelity and background consistency, and ablations validate the necessity of adaptive fusion, condition caching, and reset mechanisms. This framework enables efficient, high-fidelity, localized edits without requiring manual masks, making diffusion-based image editing more practical for real-world use-cases.

Abstract

Diffusion Transformer models have significantly advanced image editing by encoding conditional images and integrating them into transformer layers. However, most edits involve modifying only small regions, while current methods uniformly process and denoise all tokens at every timestep, causing redundant computation and potentially degrading unchanged areas. This raises a fundamental question: Is it truly necessary to regenerate every region during editing? To address this, we propose SpotEdit, a training-free diffusion editing framework that selectively updates only the modified regions. SpotEdit comprises two key components: SpotSelector identifies stable regions via perceptual similarity and skips their computation by reusing conditional image features; SpotFusion adaptively blends these features with edited tokens through a dynamic fusion mechanism, preserving contextual coherence and editing quality. By reducing unnecessary computation and maintaining high fidelity in unmodified areas, SpotEdit achieves efficient and precise image editing.

Figures (7)

• Figure 1: Reconstruction results at different timesteps.(Totalsteps T=50, seed=42, prompt = "Add a scarf to the dog.") Each reconstruction $\hat{x_0}$ is estimated following the rectified flow formulation: $\hat{X_0} = x_t - t \, v_{\theta}(x_t, c, t)$. It can be observed that some regions become sharp and visually consistent with the original image even at very early stages, while other regions continue to evolve until the final timestep.
• Figure 2: Overview of SpotEdit. The process consists of three stages: (1) Initial Steps: the model performs standard DiT denoising on all image tokens under the editing instruction, while caching the KV values for Spotfusion. (2) Spot Steps: SpotSelector dynamically identifies regenerated region and non-edited region tokens using LPIPS-like perceptual scores. Non-edited region tokens are skipped by DiT computation, while regenerated region tokens are generated iteratively with SpotFusion, which builds a temporally consistent condition cache by fusing cached non-edited region KV values with condition image KV values. (3) Token Replacement: Regenerated tokens are updated through DiT, and non-edited tokens are directly covered by the corresponding reused tokens before decoding into an image, ensuring background fidelity with reduced computation.
• Figure 3: PCA trajectories of hidden states of non-edited tokens across dual stream layers and single stream layers. As denoising progresses, the trajectory of the generated image in the non-edited region ($x$) gradually approaches that of the condition image ($y$), indicating that their latent representations progressively align. Eventually, both trajectories overlap, suggesting that unedited regions converge to the same latent subspace, thereby maintaining strong background consistency and semantic preservation.
• Figure 4: Non-edited preservation comparison across different models. Prior methods either modify unnecessary background regions or distort color consistency, whereas our method preserves non-edited areas faithfully while applying accurate edits.
• Figure 5: The qualitative ablation study on token fusion