Test-Time Detoxification without Training or Learning Anything

TL;DR

This work tackles reducing toxic outputs from large language models without retraining or gradient access. It introduces TIDE, a test-time detoxification method that perturbs prompt embeddings via zeroth-order gradient estimation to steer generations toward lower toxicity while preserving fluency. The approach is model-agnostic and requires only forward evaluations of the model and a toxicity scorer, achieving robust reductions across multiple models and prompts with a favorable toxicity–perplexity trade-off. Practically, it demonstrates that input embeddings are powerful control variables for safe text generation and highlights the potential of black-box, per-instance optimization for scalable safety in autoregressive models.

Abstract

Large language models can produce toxic or inappropriate text even for benign inputs, creating risks when deployed at scale. Detoxification is therefore important for safety and user trust, particularly when we want to reduce harmful content without sacrificing the model's generation quality. Many existing approaches rely on model retraining, gradients, or learned auxiliary components, which can be costly and may not transfer across model families or to truly black-box settings. We introduce a test-time procedure that approximates the gradient of completion toxicity with respect to the input embeddings and uses a small number of descent steps to steer generation toward less toxic continuations. This is achieved with zeroth-order optimization that requires only access to input embeddings, a toxicity scoring function, and forward evaluations of the model. Empirically, the approach delivers robust toxicity reductions across models and prompts and, in most settings, achieves the best overall toxicity-quality trade-off. More broadly, our work positions word embeddings as effective control variables and encourages wider use of black-box optimization to guide autoregressive language models toward scalable, safer text generation, without requiring any training or access to intermediate computations.

Figures (4)

• Figure 1: Illustrative example of our test-time procedure, where model completions are shown in bold. We terminate optimization once the completion toxicity $\Phi$ falls below 0.5. The procedure shifts the embedding toward regions of lower toxicity in the landscape, as shown in the second panel, where the dashed line indicates the steepest descent (theoretical optimum).
• Figure 2: Performance in terms of maximum toxicity versus fluency perplexity on prompts from the RealToxicityPrompts benchmark, averaged over three completions generated with temperature 0.1. The best performance lies in the lower left. For RAD and SASA, only the results for $\beta = \{50, 75, 100\}$ are shown for clarity, with darker markers indicating larger $\beta$ values.
• Figure 3: Performance in terms of maximum toxicity and fluency perplexity (shown separately) on prompts from the "challenging" subset of the RealToxicityPrompts benchmark, averaged over three completions generated with temperature 0.1, except in the temperature experiments. The best performance lies in the lower left. The tested values are listed in Table \ref{['tab:sensitivity_analysis_vals']}. Coral and teal represent GPT-2 and Qwen3 models, respectively, with darker colors indicating larger parameter values.
• Figure 4: Performance in terms of maximum toxicity versus fluency perplexity, averaged over three completions generated with temperature 0.1. The best performance lies in the lower left. For RAD and SASA, only the results for $\beta = \{50, 75, 100\}$ are shown for clarity, with darker markers indicating larger $\beta$ values. From left to right: Llama 3.1-8B, Qwen3-4B, Gemma 2-2B, GPT-2 Large (774M).