Avoiding Generative Model Writer's Block With Embedding Nudging

TL;DR

Problem: diffusion-based image generation risks memorization of training data. Approach: inference-time latent-space manipulation that pulls toward desired embeddings and pushes away from undesired ones, guided by an adaptive sequence of quality filters. Key contribution: a gradient-descent-based push/pull objective on latent $L$ with $\min_{L} -\alpha \frac{\sum_{D \in I_D}{D \times L}}{|I_D|} + \beta \frac{\sum_{U \in I_U}{U \times L}}{|I_U|} + |L-L_0|$, enabling selective memorization suppression without retraining. Findings: experiments on Stable Diffusion 1.4 demonstrate avoidance of memorized training images with comparable image quality, validated by embedding analyses and human studies. Significance: provides a practical path to privacy-preserving, controllable image generation in latent diffusion models.

Abstract

Generative image models, since introduction, have become a global phenomenon. From new arts becoming possible to new vectors of abuse, many new capabilities have become available. One of the challenging issues with generative models is controlling the generation process specially to prevent specific generations classes or instances . There are several reasons why one may want to control the output of generative models, ranging from privacy and safety concerns to application limitations or user preferences To address memorization and privacy challenges, there has been considerable research dedicated to filtering prompts or filtering the outputs of these models. What all these solutions have in common is that at the end of the day they stop the model from producing anything, hence limiting the usability of the model. In this paper, we propose a method for addressing this usability issue by making it possible to steer away from unwanted concepts (when detected in model's output) and still generating outputs. In particular we focus on the latent diffusion image generative models and how one can prevent them to generate particular images while generating similar images with limited overhead. We focus on mitigating issues like image memorization, demonstrating our technique's effectiveness through qualitative and quantitative evaluations. Our method successfully prevents the generation of memorized training images while maintaining comparable image quality and relevance to the unmodified model.

Figures (6)

• Figure 1: Overview of the methodology. We optimize the latent representation of the diffusion model to avoid generating unwanted outputs.
• Figure 2: Our adaptive image generation system uses a series of filters to assess output quality. If filters are triggered, the system dynamically selects points to adjust the generation and improve it. This approach only adds computational cost when filters identify potential issues, leaving unproblematic prompts unmodified.
• Figure 3: Our approach prevents memorization of training images, generating similar yet distinct images from the same captions. This maintains image relevance while mitigating overfitting concerns.
• Figure 4: Distrubiton of the l2 distance between the generated images and the target images
• Figure 5: Arrows representing the changes from unmodified image generation to the modified versions using our approach in 2D projection of CLIP embeddings.