Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions

TL;DR

This work tackles NSFW content and social bias in diffusion-based image synthesis by learning a continuous semantic direction in the CLIP embedding space that confines the entire text prompt to a safe region. The direction is discovered via optimization over the diffusion denoising process using implicit classifier signals and initialized with Low-Rank Adaptation (LoRA) to minimize side effects on other semantics. The approach enables safe and fair generation at inference time by augmenting the prompt embedding with a learned direction and can be combined with existing methods to enhance responsible generation. Empirical results on NSFW reduction, bias mitigation, and cross-model transferability demonstrate strong practical impact while preserving image fidelity and text alignment.

Abstract

The remarkable ability of diffusion models to generate high-fidelity images has led to their widespread adoption. However, concerns have also arisen regarding their potential to produce Not Safe for Work (NSFW) content and exhibit social biases, hindering their practical use in real-world applications. In response to this challenge, prior work has focused on employing security filters to identify and exclude toxic text, or alternatively, fine-tuning pre-trained diffusion models to erase sensitive concepts. Unfortunately, existing methods struggle to achieve satisfactory performance in the sense that they can have a significant impact on the normal model output while still failing to prevent the generation of harmful content in some cases. In this paper, we propose a novel self-discovery approach to identifying a semantic direction vector in the embedding space to restrict text embedding within a safe region. Our method circumvents the need for correcting individual words within the input text and steers the entire text prompt towards a safe region in the embedding space, thereby enhancing model robustness against all possibly unsafe prompts. In addition, we employ Low-Rank Adaptation (LoRA) for semantic direction vector initialization to reduce the impact on the model performance for other semantics. Furthermore, our method can also be integrated with existing methods to improve their social responsibility. Extensive experiments on benchmark datasets demonstrate that our method can effectively reduce NSFW content and mitigate social bias generated by diffusion models compared to several state-of-the-art baselines.

Figures (12)

• Figure 1: Illustration of our method that utilizes the disparities in diffusion noise distribution to identify semantic directions in the CLIP embedding space to guide the generation process and avoid inappropriate content.
• Figure 2: Examples of using two contrasting prompts to identify specific semantic directions. The images in each column are generated with the same prompt and seed.
• Figure 3: Illustration of the optimization process to find a direction vector associated with the target concept in the CLIP embedding space. The noise distribution close to or far away from the target concept is obtained through the frozen pre-trained diffusion model. The $l_2$-loss between $\epsilon_{\theta}\left( \mathbf{z}_{t}, c + d, t \right)$ and $\psi(\mathbf{z}_t,\mathbf{c}_o,t)$ in Eq. \ref{['eq:psi']} at each step $t$ makes the noise predicted by the base prompt with the direction vector added close to the noise distribution. The updated direction vector $d'$ are used in the next step of denoising, and the precise direction is learned in the iterative denoising process.
• Figure 4: Illustration of two methods for direction vector initialization. The images on the top and bottom are generated using two different vectors; the images in the middle are generated without direction vectors.
• Figure 5: Illustration of our method and baselines for reducing inappropriate content in image generation. Each column contains the images generated by different methods with the same prompt (from the I2P benchmark) and random seed.