Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Training-Free Disentangled Text-Guided Image Editing via Sparse Latent Constraints

Mutiara Shabrina, Nova Kurnia Putri, Jefri Satria Ferdiansyah, Sabita Khansa Dewi, Novanto Yudistira

TL;DR

This work analyzes the Predict, Prevent, and Evaluate PPE framework for text-driven image editing and identifies that L2 regularization induces dense, entangled latent updates. It proposes a sparsity-based latent constraint with ultra-strict layer masking to localize edits to mid-level StyleGAN2 layers, thereby preserving identity while applying targeted attributes such as bangs. Experiments on CelebA-HQ show reduced non-target changes and improved disentanglement compared to the baseline PPE, supported by both qualitative and quantitative metrics. The approach emphasizes structured latent constraints as a practical path toward more reliable and identity-preserving text-driven manipulation.

Abstract

Text-driven image manipulation often suffers from attribute entanglement, where modifying a target attribute (e.g., adding bangs) unintentionally alters other semantic properties such as identity or appearance. The Predict, Prevent, and Evaluate (PPE) framework addresses this issue by leveraging pre-trained vision-language models for disentangled editing. In this work, we analyze the PPE framework, focusing on its architectural components, including BERT-based attribute prediction and StyleGAN2-based image generation on the CelebA-HQ dataset. Through empirical analysis, we identify a limitation in the original regularization strategy, where latent updates remain dense and prone to semantic leakage. To mitigate this issue, we introduce a sparsity-based constraint using L1 regularization on latent space manipulation. Experimental results demonstrate that the proposed approach enforces more focused and controlled edits, effectively reducing unintended changes in non-target attributes while preserving facial identity.

Training-Free Disentangled Text-Guided Image Editing via Sparse Latent Constraints

TL;DR

This work analyzes the Predict, Prevent, and Evaluate PPE framework for text-driven image editing and identifies that L2 regularization induces dense, entangled latent updates. It proposes a sparsity-based latent constraint with ultra-strict layer masking to localize edits to mid-level StyleGAN2 layers, thereby preserving identity while applying targeted attributes such as bangs. Experiments on CelebA-HQ show reduced non-target changes and improved disentanglement compared to the baseline PPE, supported by both qualitative and quantitative metrics. The approach emphasizes structured latent constraints as a practical path toward more reliable and identity-preserving text-driven manipulation.

Abstract

Text-driven image manipulation often suffers from attribute entanglement, where modifying a target attribute (e.g., adding bangs) unintentionally alters other semantic properties such as identity or appearance. The Predict, Prevent, and Evaluate (PPE) framework addresses this issue by leveraging pre-trained vision-language models for disentangled editing. In this work, we analyze the PPE framework, focusing on its architectural components, including BERT-based attribute prediction and StyleGAN2-based image generation on the CelebA-HQ dataset. Through empirical analysis, we identify a limitation in the original regularization strategy, where latent updates remain dense and prone to semantic leakage. To mitigate this issue, we introduce a sparsity-based constraint using L1 regularization on latent space manipulation. Experimental results demonstrate that the proposed approach enforces more focused and controlled edits, effectively reducing unintended changes in non-target attributes while preserving facial identity.
Paper Structure (25 sections, 3 equations, 6 figures, 3 tables)

This paper contains 25 sections, 3 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Theoretical Foundation and Baseline Analysis
  3. The PPE Framework
  4. Predict
  5. Prevent
  6. Evaluate
  7. StyleGAN2 Latent Space Architecture
  8. Analysis of Baseline Failure
  9. Methodology: The Original Framework
  10. Architecture and Implementation
  11. Dataset and Pre-processing
  12. Original Manipulation Formulation
  13. Proposed Improvement
  14. Latent Sparsity Regularization
  15. Ultra-Strict Layer Masking
  16. ...and 10 more sections

Figures (6)

  • Figure 1: Quantitative analysis of the original PPE model. High L1 values and large L2 magnitudes indicate dense and unstable latent updates.
  • Figure 2: Quantitative analysis of the improved PPE model. Reduced and stable L1 and L2 values demonstrate sparse and efficient latent manipulation.
  • Figure 3: Layer-wise analysis of the original PPE model. Significant changes occur in non-target attributes, indicating severe entanglement.
  • Figure 4: Layer-wise analysis of the improved PPE model. Non-target attributes are effectively suppressed while the target attribute remains active.
  • Figure 5: Qualitative comparison on a male subject when adding bangs. Left: result from the original PPE model exhibiting unintended makeup artifacts and identity shifts. Right: result from the improved PPE model, where the target attribute is applied while preserving facial identity.
  • ...and 1 more figures