Cleansing the Artificial Mind: A Self-Reflective Detoxification Framework for Large Language Models

TL;DR

The paper tackles toxic content generation in Large Language Models by proposing Self-Reflective Detoxification (SRD), a fully self-contained framework that uses a Toxic Signal Detector, a self-generated Contrastive Dataset via Self-Reflection, and Direct Preference Optimization (DPO) to detoxify models without external modules or human data. SRD builds an internal signal list, iteratively rewrites toxic outputs into non-toxic versions, and fine-tunes the model on a contrastive dataset, achieving strong detoxification while preserving or even improving performance on general tasks like $MMLU$ and $GSM8K$. Empirical results on DetoxLLM and ParaDetox benchmarks show SRD often outperforms state‑of‑the‑art detox methods in toxicity metrics such as $MTV$, $T5MTV$, and $T.R.$, with competitive perplexity ($PPL$) and high semantic fluency ($SIM$, $STA$, $FL$, $J$-Score). The findings demonstrate the viability of truly self-regulated LLMs, offering scalable, ethically guided text generation with reduced reliance on external tools or human labor, and they chart a path toward more responsible AI deployment.

Abstract

Recent breakthroughs in Large Language Models (LLMs) have revealed remarkable generative capabilities and emerging self-regulatory mechanisms, including self-correction and self-rewarding. However, current detoxification techniques rarely exploit these built-in abilities; instead, they rely on external modules, labor-intensive data annotation, or human intervention --factors that hinder scalability and consistency. In this paper, we introduce a fully self-reflective detoxification framework that harnesses the inherent capacities of LLMs to detect, correct toxic content, and refine LLMs without external modules and data annotation. Specifically, we propose a Toxic Signal Detector --an internal self-identification mechanism, coupled with a systematic intervention process to transform toxic text into its non-toxic counterpart. This iterative procedure yields a contrastive detoxification dataset used to fine-tune the model, enhancing its ability for safe and coherent text generation. Experiments on benchmark datasets such as DetoxLLM and ParaDetox show that our method achieves better detoxification performance than state-of-the-art methods while preserving semantic fidelity. By obviating the need for human intervention or external components, this paper reveals the intrinsic self-detoxification ability of LLMs, offering a consistent and effective approach for mitigating harmful content generation. Ultimately, our findings underscore the potential for truly self-regulated language models, paving the way for more responsible and ethically guided text generation systems.

Figures (18)

• Figure 1: Overview of the Self-Reflective Detoxification (SRD) framework. The process involves building a signal list through self-construction, generating a contrastive dataset through self-reflection, and fine-tuning the model.
• Figure 2: Probability density function (PDF) of sentence toxicity values for Group I and Group II. Group I: Sentences containing newly generated words match entries in the signal list. Group II: Sentences with newly generated words are not found in the signal list. The black dashed line marks a 50% toxicity value threshold.
• Figure 3: The relationship between Signal List Length and Toxic Ratio(T.R.).
• Figure 4: The prompt -- the ability of model to detect toxicity
• Figure 5: The output obtained by inputting "Toxic Content" into Llama-3.1-8B-Instruct.