Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards SFW sampling for diffusion models via external conditioning

Camilo Carvajal Reyes, Joaquín Fontbona, Felipe Tobar

TL;DR

The paper tackles the risk of NSFW content in diffusion-based image synthesis by introducing an external safety mechanism that relies on a harmfulness density $p_h$ defined via CLIP-based multimodal embeddings. It combines manifold-preserving diffusion guidance with a Conditional Trajectory Correction (CTC) to steer samples away from high-harmful regions while preserving image quality, particularly in latent diffusion settings. Empirical results on Stable Diffusion show meaningful reductions in explicit content detection with varying trade-offs in prompt fidelity and aesthetic quality, demonstrating the viability of model-agnostic safety via external conditioning. This work highlights the potential of external, multimodal signals for aligning SBMs toward safer outputs and suggests avenues for further integration with model fine-tuning and multiple external regulators.

Abstract

Score-based generative models (SBM), also known as diffusion models, are the de facto state of the art for image synthesis. Despite their unparalleled performance, SBMs have recently been in the spotlight for being tricked into creating not-safe-for-work (NSFW) content, such as violent images and non-consensual nudity. Current approaches that prevent unsafe generation are based on the models' own knowledge, and the majority of them require fine-tuning. This article explores the use of external sources for ensuring safe outputs in SBMs. Our safe-for-work (SFW) sampler implements a Conditional Trajectory Correction step that guides the samples away from undesired regions in the ambient space using multimodal models as the source of conditioning. Furthermore, using Contrastive Language Image Pre-training (CLIP), our method admits user-defined NSFW classes, which can vary in different settings. Our experiments on the text-to-image SBM Stable Diffusion validate that the proposed SFW sampler effectively reduces the generation of explicit content while being competitive with other fine-tuning-based approaches, as assessed via independent NSFW detectors. Moreover, we evaluate the impact of the SFW sampler on image quality and show that the proposed correction scheme comes at a minor cost with negligible effect on samples not needing correction. Our study confirms the suitability of the SFW sampler towards aligned SBM models and the potential of using model-agnostic conditioning for the prevention of unwanted images.

Towards SFW sampling for diffusion models via external conditioning

TL;DR

The paper tackles the risk of NSFW content in diffusion-based image synthesis by introducing an external safety mechanism that relies on a harmfulness density defined via CLIP-based multimodal embeddings. It combines manifold-preserving diffusion guidance with a Conditional Trajectory Correction (CTC) to steer samples away from high-harmful regions while preserving image quality, particularly in latent diffusion settings. Empirical results on Stable Diffusion show meaningful reductions in explicit content detection with varying trade-offs in prompt fidelity and aesthetic quality, demonstrating the viability of model-agnostic safety via external conditioning. This work highlights the potential of external, multimodal signals for aligning SBMs toward safer outputs and suggests avenues for further integration with model fine-tuning and multiple external regulators.

Abstract

Score-based generative models (SBM), also known as diffusion models, are the de facto state of the art for image synthesis. Despite their unparalleled performance, SBMs have recently been in the spotlight for being tricked into creating not-safe-for-work (NSFW) content, such as violent images and non-consensual nudity. Current approaches that prevent unsafe generation are based on the models' own knowledge, and the majority of them require fine-tuning. This article explores the use of external sources for ensuring safe outputs in SBMs. Our safe-for-work (SFW) sampler implements a Conditional Trajectory Correction step that guides the samples away from undesired regions in the ambient space using multimodal models as the source of conditioning. Furthermore, using Contrastive Language Image Pre-training (CLIP), our method admits user-defined NSFW classes, which can vary in different settings. Our experiments on the text-to-image SBM Stable Diffusion validate that the proposed SFW sampler effectively reduces the generation of explicit content while being competitive with other fine-tuning-based approaches, as assessed via independent NSFW detectors. Moreover, we evaluate the impact of the SFW sampler on image quality and show that the proposed correction scheme comes at a minor cost with negligible effect on samples not needing correction. Our study confirms the suitability of the SFW sampler towards aligned SBM models and the potential of using model-agnostic conditioning for the prevention of unwanted images.
Paper Structure (20 sections, 6 equations, 2 figures, 3 tables)

This paper contains 20 sections, 6 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Preliminary concepts on diffusion models.
  4. Non-Markovian sampling.
  5. Manifold-preserving sampler
  6. Contrastive language image pre-training (CLIP)
  7. Safe-for-work sampling
  8. Harmfulness mitigation via manifold-preserving sampling
  9. Conditional trajectory correction
  10. CLIP-based construction of the harmfulness density $p_h$
  11. Related works
  12. Works tackling NSFW generation.
  13. Connection with negative classifier guidance.
  14. Experiments
  15. Assessing the ability to mitigate NSFW content
  16. ...and 5 more sections

Figures (2)

  • Figure 1: Illustration of the proposed gradient-based correction conditional to the assessment of an external harmful classifier.
  • Figure 2: Examples of image generations using SFW sampling. On the left most column we provide the text prompt used for sampling, followed by the original sample using Stable Diffusion without correction. We then show examples for the same prompt and seed using the three investigated variants mention in Sec. \ref{['sec:experiments']}.