Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VENUS: Visual Editing with Noise Inversion Using Scene Graphs

Thanh-Nhan Vo, Trong-Thuan Nguyen, Tam V. Nguyen, Minh-Triet Tran

TL;DR

This work introduces VENUS (Visual Editing with Noise inversion Using Scene graphs), a training-free framework for scene graph-guided image editing that employs a split prompt conditioning strategy that disentangles the target object of the edit from its background context, while simultaneously leveraging noise inversion to preserve fidelity in unedited regions.

Abstract

State-of-the-art text-based image editing models often struggle to balance background preservation with semantic consistency, frequently resulting either in the synthesis of entirely new images or in outputs that fail to realize the intended edits. In contrast, scene graph-based image editing addresses this limitation by providing a structured representation of semantic entities and their relations, thereby offering improved controllability. However, existing scene graph editing methods typically depend on model fine-tuning, which incurs high computational cost and limits scalability. To this end, we introduce VENUS (Visual Editing with Noise inversion Using Scene graphs), a training-free framework for scene graph-guided image editing. Specifically, VENUS employs a split prompt conditioning strategy that disentangles the target object of the edit from its background context, while simultaneously leveraging noise inversion to preserve fidelity in unedited regions. Moreover, our proposed approach integrates scene graphs extracted from multimodal large language models with diffusion backbones, without requiring any additional training. Empirically, VENUS substantially improves both background preservation and semantic alignment on PIE-Bench, increasing PSNR from 22.45 to 24.80, SSIM from 0.79 to 0.84, and reducing LPIPS from 0.100 to 0.070 relative to the state-of-the-art scene graph editing model (SGEdit). In addition, VENUS enhances semantic consistency as measured by CLIP similarity (24.97 vs. 24.19). On EditVal, VENUS achieves the highest fidelity with a 0.87 DINO score and, crucially, reduces per-image runtime from 6-10 minutes to only 20-30 seconds. Beyond scene graph-based editing, VENUS also surpasses strong text-based editing baselines such as LEDIT++ and P2P+DirInv, thereby demonstrating consistent improvements across both paradigms.

VENUS: Visual Editing with Noise Inversion Using Scene Graphs

TL;DR

This work introduces VENUS (Visual Editing with Noise inversion Using Scene graphs), a training-free framework for scene graph-guided image editing that employs a split prompt conditioning strategy that disentangles the target object of the edit from its background context, while simultaneously leveraging noise inversion to preserve fidelity in unedited regions.

Abstract

State-of-the-art text-based image editing models often struggle to balance background preservation with semantic consistency, frequently resulting either in the synthesis of entirely new images or in outputs that fail to realize the intended edits. In contrast, scene graph-based image editing addresses this limitation by providing a structured representation of semantic entities and their relations, thereby offering improved controllability. However, existing scene graph editing methods typically depend on model fine-tuning, which incurs high computational cost and limits scalability. To this end, we introduce VENUS (Visual Editing with Noise inversion Using Scene graphs), a training-free framework for scene graph-guided image editing. Specifically, VENUS employs a split prompt conditioning strategy that disentangles the target object of the edit from its background context, while simultaneously leveraging noise inversion to preserve fidelity in unedited regions. Moreover, our proposed approach integrates scene graphs extracted from multimodal large language models with diffusion backbones, without requiring any additional training. Empirically, VENUS substantially improves both background preservation and semantic alignment on PIE-Bench, increasing PSNR from 22.45 to 24.80, SSIM from 0.79 to 0.84, and reducing LPIPS from 0.100 to 0.070 relative to the state-of-the-art scene graph editing model (SGEdit). In addition, VENUS enhances semantic consistency as measured by CLIP similarity (24.97 vs. 24.19). On EditVal, VENUS achieves the highest fidelity with a 0.87 DINO score and, crucially, reduces per-image runtime from 6-10 minutes to only 20-30 seconds. Beyond scene graph-based editing, VENUS also surpasses strong text-based editing baselines such as LEDIT++ and P2P+DirInv, thereby demonstrating consistent improvements across both paradigms.
Paper Structure (24 sections, 7 equations, 3 figures, 6 tables)

This paper contains 24 sections, 7 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Scene Graph to Image Synthesis and Editing
  4. LLM-based Image Synthesis
  5. Diffusion-Based Image Editing
  6. Discussion
  7. Problem Formulation
  8. Our Proposed Approach
  9. Scene Graph Construction and Refinement via MLLM
  10. Target prompt and source prompt construction
  11. Target Prompt Construction.
  12. Source Prompt Construction
  13. Image Editing
  14. Scene graph-based editing
  15. Text-based editing
  16. ...and 9 more sections

Figures (3)

  • Figure 1: An illustration of our VENUS approach. A scene graph is constructed from the input image, then edited by either the MLLM or the user. The edited graph is parsed into structured prompts, which condition a frozen diffusion model to edit the image. (Best viewed in color and with zoom.)
  • Figure 2: Examples of scene graph guided image editing. Top row: changing the horse into a zebra by updating the corresponding node in the scene graph. Bottom row: removing the moon in the background by deleting its associated nodes and relations. (Best viewed in color and with zoom.)
  • Figure 3: Comparison of editing results across a variety of tasks. Compared to LEDIT++, P2P-DirInv, and PnP-DirInv, VENUS (ours) produces edits that are both semantically accurate and visually consistent. (Best viewed in color and with zoom.)