Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FusionEdit: Semantic Fusion and Attention Modulation for Training-Free Image Editing

Yongwen Lai, Chaoqun Wang, Shaobo Min

TL;DR

FusionEdit addresses the challenge of training-free, precise text guided image editing by automatically localizing edits through semantic discrepancy between source and target prompts. It introduces a distance aware soft mask via region growing and total variation regularization to create smooth boundaries, enabling natural edits without hard mask artifacts. Disparity aware attention modulation (DAM) via AdaIN injects global source statistics into the masked editing path, preserving global coherence while maintaining local editability. Extensive experiments on PIE-Bench demonstrate state-of-the-art performance, validating both the soft boundary fusion and DAM components for robust, controllable editing without extra supervision.

Abstract

Text-guided image editing aims to modify specific regions according to the target prompt while preserving the identity of the source image. Recent methods exploit explicit binary masks to constrain editing, but hard mask boundaries introduce artifacts and reduce editability. To address these issues, we propose FusionEdit, a training-free image editing framework that achieves precise and controllable edits. First, editing and preserved regions are automatically identified by measuring semantic discrepancies between source and target prompts. To mitigate boundary artifacts, FusionEdit performs distance-aware latent fusion along region boundaries to yield the soft and accurate mask, and employs a total variation loss to enforce smooth transitions, obtaining natural editing results. Second, FusionEdit leverages AdaIN-based modulation within DiT attention layers to perform a statistical attention fusion in the editing region, enhancing editability while preserving global consistency with the source image. Extensive experiments demonstrate that our FusionEdit significantly outperforms state-of-the-art methods. Code is available at \href{https://github.com/Yvan1001/FusionEdit}{https://github.com/Yvan1001/FusionEdit}.

FusionEdit: Semantic Fusion and Attention Modulation for Training-Free Image Editing

TL;DR

FusionEdit addresses the challenge of training-free, precise text guided image editing by automatically localizing edits through semantic discrepancy between source and target prompts. It introduces a distance aware soft mask via region growing and total variation regularization to create smooth boundaries, enabling natural edits without hard mask artifacts. Disparity aware attention modulation (DAM) via AdaIN injects global source statistics into the masked editing path, preserving global coherence while maintaining local editability. Extensive experiments on PIE-Bench demonstrate state-of-the-art performance, validating both the soft boundary fusion and DAM components for robust, controllable editing without extra supervision.

Abstract

Text-guided image editing aims to modify specific regions according to the target prompt while preserving the identity of the source image. Recent methods exploit explicit binary masks to constrain editing, but hard mask boundaries introduce artifacts and reduce editability. To address these issues, we propose FusionEdit, a training-free image editing framework that achieves precise and controllable edits. First, editing and preserved regions are automatically identified by measuring semantic discrepancies between source and target prompts. To mitigate boundary artifacts, FusionEdit performs distance-aware latent fusion along region boundaries to yield the soft and accurate mask, and employs a total variation loss to enforce smooth transitions, obtaining natural editing results. Second, FusionEdit leverages AdaIN-based modulation within DiT attention layers to perform a statistical attention fusion in the editing region, enhancing editability while preserving global consistency with the source image. Extensive experiments demonstrate that our FusionEdit significantly outperforms state-of-the-art methods. Code is available at \href{https://github.com/Yvan1001/FusionEdit}{https://github.com/Yvan1001/FusionEdit}.
Paper Structure (11 sections, 11 equations, 6 figures, 2 tables)

This paper contains 11 sections, 11 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Method
  4. Adaptive Soft Mask Generation
  5. Disparity-Aware Attention Modulation
  6. Experimental Results
  7. Experimental Settings
  8. Comparison with State-of-the-art Methods
  9. Ablation Study
  10. Conclusion
  11. Acknowledgments

Figures (6)

  • Figure 1: Overview of the FusionEdit pipeline. Given a source image with source and target prompts, FusionEdit generates the semantic discrepancy map from target- and source-conditioned velocity fields to produce an adaptive soft mask that guides localized editing. Disparity-aware attention modulation (DAM) further injects global appearance statistics from the unmasked stream into the masked editing path, enabling precise and consistent image editing.
  • Figure 2: Qualitative comparisons on text-guided image editing. Our FusionEdit achieves faithful modifications according to the target prompt while preserving essential content of the source image.
  • Figure 3: User study results. Our FusionEdit receives the highest number of user selections.
  • Figure 4: Comparison of editing results with binary mask vs. soft mask.
  • Figure 5: Visualization of semantic discrepancy map and soft mask.
  • ...and 1 more figures