Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MDE-Edit: Masked Dual-Editing for Multi-Object Image Editing via Diffusion Models

Hongyang Zhu, Haipeng Liu, Bo Fu, Yang Wang

TL;DR

Multi-object image editing with diffusion models faces localization and attribute-binding challenges in complex scenes. The authors introduce MDE-Edit, a training-free framework that performs inference-time latent optimization using a masked dual-edit strategy governed by Object Alignment Loss (OAL) and Color Consistency Loss (CCL) to separate object structure from appearance changes. This decoupling yields precise object localization and region-confined attribute edits, effectively mitigating attention misalignment and leakage in overlapping objects. Extensive experiments on diverse benchmarks demonstrate state-of-the-art editing accuracy and visual fidelity, indicating strong practical potential for robust multi-object manipulation in real-world images.

Abstract

Multi-object editing aims to modify multiple objects or regions in complex scenes while preserving structural coherence. This task faces significant challenges in scenarios involving overlapping or interacting objects: (1) Inaccurate localization of target objects due to attention misalignment, leading to incomplete or misplaced edits; (2) Attribute-object mismatch, where color or texture changes fail to align with intended regions due to cross-attention leakage, creating semantic conflicts (\textit{e.g.}, color bleeding into non-target areas). Existing methods struggle with these challenges: approaches relying on global cross-attention mechanisms suffer from attention dilution and spatial interference between objects, while mask-based methods fail to bind attributes to geometrically accurate regions due to feature entanglement in multi-object scenarios. To address these limitations, we propose a training-free, inference-stage optimization approach that enables precise localized image manipulation in complex multi-object scenes, named MDE-Edit. MDE-Edit optimizes the noise latent feature in diffusion models via two key losses: Object Alignment Loss (OAL) aligns multi-layer cross-attention with segmentation masks for precise object positioning, and Color Consistency Loss (CCL) amplifies target attribute attention within masks while suppressing leakage to adjacent regions. This dual-loss design ensures localized and coherent multi-object edits. Extensive experiments demonstrate that MDE-Edit outperforms state-of-the-art methods in editing accuracy and visual quality, offering a robust solution for complex multi-object image manipulation tasks.

MDE-Edit: Masked Dual-Editing for Multi-Object Image Editing via Diffusion Models

TL;DR

Multi-object image editing with diffusion models faces localization and attribute-binding challenges in complex scenes. The authors introduce MDE-Edit, a training-free framework that performs inference-time latent optimization using a masked dual-edit strategy governed by Object Alignment Loss (OAL) and Color Consistency Loss (CCL) to separate object structure from appearance changes. This decoupling yields precise object localization and region-confined attribute edits, effectively mitigating attention misalignment and leakage in overlapping objects. Extensive experiments on diverse benchmarks demonstrate state-of-the-art editing accuracy and visual fidelity, indicating strong practical potential for robust multi-object manipulation in real-world images.

Abstract

Multi-object editing aims to modify multiple objects or regions in complex scenes while preserving structural coherence. This task faces significant challenges in scenarios involving overlapping or interacting objects: (1) Inaccurate localization of target objects due to attention misalignment, leading to incomplete or misplaced edits; (2) Attribute-object mismatch, where color or texture changes fail to align with intended regions due to cross-attention leakage, creating semantic conflicts (\textit{e.g.}, color bleeding into non-target areas). Existing methods struggle with these challenges: approaches relying on global cross-attention mechanisms suffer from attention dilution and spatial interference between objects, while mask-based methods fail to bind attributes to geometrically accurate regions due to feature entanglement in multi-object scenarios. To address these limitations, we propose a training-free, inference-stage optimization approach that enables precise localized image manipulation in complex multi-object scenes, named MDE-Edit. MDE-Edit optimizes the noise latent feature in diffusion models via two key losses: Object Alignment Loss (OAL) aligns multi-layer cross-attention with segmentation masks for precise object positioning, and Color Consistency Loss (CCL) amplifies target attribute attention within masks while suppressing leakage to adjacent regions. This dual-loss design ensures localized and coherent multi-object edits. Extensive experiments demonstrate that MDE-Edit outperforms state-of-the-art methods in editing accuracy and visual quality, offering a robust solution for complex multi-object image manipulation tasks.
Paper Structure (14 sections, 8 equations, 6 figures, 2 tables)

This paper contains 14 sections, 8 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminary
  4. Methodology
  5. Attention Injection
  6. Masked Dual-Edit via Attention
  7. Object Alignment Loss (OAL)
  8. Color Consistency Loss (CCL)
  9. Latent Variable Optimization
  10. Experiments
  11. Experimental Details
  12. Comparison with State-of-the-arts
  13. Ablation Studies
  14. Conclusion

Figures (6)

  • Figure 1: Illustration of our proposed MDE-Edit framework. (a) The proposed MDE-Edit framework consists of two main branches: the upper row depicts the reconstruction branch, while the lower row represents the editing branch. Within the editing branch, the noise latent feature undergoes optimization through the MDE process. This optimization mechanism facilitates precise alignment with the specified edit prompts, namely “dog” and “red hat”, while simultaneously maintaining focus within the designated edit region masks. (b) Illustration of MDE optimization. OAL and CCL respectively handle the positioning of the object and the binding of attributes to the object.
  • Figure 2: Multi-object editing failure cases. Revealing challenges in maintaining coherent attribute-object associations and preventing unintended hybrid outputs.
  • Figure 3: Qualitative comparison with the state-of-the-arts in simple multi-object scenes without overlap. While other approaches struggle with incomplete edits, unintended modifications, or deviations from the prompt, MDE-Edit reliably edits the image as specified.
  • Figure 4: Qualitative comparison with state-of-the-art methods in multi-object overlapping scenarios. In complex scenarios involving multi-object overlap, existing methods often produce suboptimal results—either failing to modify the intended regions, distorting overlapping elements, or introducing inconsistencies with the target description. MDE-Edit, however, demonstrates exceptional precision in such challenging conditions, seamlessly applying edits only to specified objects while preserving overlapping and adjacent areas without interference.
  • Figure 5: Multi-object editing results of MDE-Edit. Showcasing the diverse multi-object editing capabilities of MDE-Edit, featuring varied subjects (humans, animals, crafted objects) and contexts (natural scenes, imaginative scenarios), highlighting the method's adaptability across styles and compositions.
  • ...and 1 more figures