DTVI: Dual-Stage Textual and Visual Intervention for Safe Text-to-Image Generation

Abstract

Text-to-Image (T2I) diffusion models have demonstrated strong generation ability, but their potential to generate unsafe content raises significant safety concerns. Existing inference-time defense methods typically perform category-agnostic token-level intervention in the text embedding space, which fails to capture malicious semantics distributed across the full token sequence and remains vulnerable to adversarial prompts. In this paper, we propose DTVI, a dual-stage inference-time defense framework for safe T2I generation. Unlike existing methods that intervene on specific token embeddings, our method introduces category-aware sequence-level intervention on the full prompt embedding to better capture distributed malicious semantics, and further attenuates the remaining unsafe influences during the visual generation stage. Experimental results on real-world unsafe prompts, adversarial prompts, and multiple harmful categories show that our method achieves effective and robust defense while preserving reasonable generation quality on benign prompts, obtaining an average Defense Success Rate (DSR) of 94.43% across sexual-category benchmarks and 88.56 across seven unsafe categories, while maintaining generation quality on benign prompts.

Figures (5)

• Figure 1: Overall pipeline of DTVI. Given a user prompt, it performs a dual-stage inference-time intervention to suppress unsafe generation while preserving benign intent. In the textual stage, the prompt embedding is purified by removing unsafe semantic components and steering it away from the unsafe region. In the visual stage, unsafe visual tendencies are further suppressed during denoising by reducing feature responses aligned with the visual steering direction, thus improving safety with minimal disruption to benign generation.
• Figure 2: Some examples of negative defense cases on ESD, UCE, and SafeGen. In these cases, the defended models generate images with more explicit unsafe content than the undefended model, leading to negative DSR values.
• Figure 3: Parameter sensitivity analysis. Effect of varying $\lambda$ (left) and $\epsilon_f$ (right) on defense performance (DSR, left axis) and image quality (FID, right axis). The higher DSR indicates stronger defense, lower FID indicates better image fidelity.
• Figure 4: We use NudeNet nudenet to obtain average nude scores under single-module ablations on three representative unsafe benchmarks. Both modules reduce unsafe visual content compared with the undefended setting.
• Figure 5: Qualitative comparison in the ablation studies. Columns correspond to the undefended model, textual-only intervention, visual-only intervention, and joint textual-visual intervention. The joint setting produces safer and less explicit outputs than using either intervention alone.