Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective

Zhiqiang Kou, Junyang Chen, Xin-Qiang Cai, Ming-Kun Xie, Biao Liu, Changwei Wang, Lei Feng, Yuheng Jia, Gang Niu, Masashi Sugiyama, Xin Geng

TL;DR

This work tackles the problem that toxicity in LLMs is inherently multi-label and often misrepresented by single-label benchmarks. It introduces three unified 15-category benchmarks—Q-A-MLL, R-A-MLL, H-X-MLL—for robust, fine-grained toxicity evaluation and builds LEPL-MLL, a pseudo-labeling framework that leverages partial supervision, contrastive label enhancement, and a GCN to model label correlations. A key theoretical contribution shows that, under favorable unreliability degrees, pseudo-labels can achieve lower risk and better sample efficiency than traditional single-label learning. Empirically, LEPL-MLL outperforms strong baselines including GPT‑4o and DeepSeek across all benchmarks and remains effective under reduced label coverage, enabling scalable, cost-efficient LLM safety alignment with practical impact for deployment and evaluation.

Abstract

Large language models (LLMs) have achieved impressive results across a range of natural language processing tasks, but their potential to generate harmful content has raised serious safety concerns. Current toxicity detectors primarily rely on single-label benchmarks, which cannot adequately capture the inherently ambiguous and multi-dimensional nature of real-world toxic prompts. This limitation results in biased evaluations, including missed toxic detections and false positives, undermining the reliability of existing detectors. Additionally, gathering comprehensive multi-label annotations across fine-grained toxicity categories is prohibitively costly, further hindering effective evaluation and development. To tackle these issues, we introduce three novel multi-label benchmarks for toxicity detection: \textbf{Q-A-MLL}, \textbf{R-A-MLL}, and \textbf{H-X-MLL}, derived from public toxicity datasets and annotated according to a detailed 15-category taxonomy. We further provide a theoretical proof that, on our released datasets, training with pseudo-labels yields better performance than directly learning from single-label supervision. In addition, we develop a pseudo-label-based toxicity detection method. Extensive experimental results show that our approach significantly surpasses advanced baselines, including GPT-4o and DeepSeek, thus enabling more accurate and reliable evaluation of multi-label toxicity in LLM-generated content.

Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective

TL;DR

This work tackles the problem that toxicity in LLMs is inherently multi-label and often misrepresented by single-label benchmarks. It introduces three unified 15-category benchmarks—Q-A-MLL, R-A-MLL, H-X-MLL—for robust, fine-grained toxicity evaluation and builds LEPL-MLL, a pseudo-labeling framework that leverages partial supervision, contrastive label enhancement, and a GCN to model label correlations. A key theoretical contribution shows that, under favorable unreliability degrees, pseudo-labels can achieve lower risk and better sample efficiency than traditional single-label learning. Empirically, LEPL-MLL outperforms strong baselines including GPT‑4o and DeepSeek across all benchmarks and remains effective under reduced label coverage, enabling scalable, cost-efficient LLM safety alignment with practical impact for deployment and evaluation.

Abstract

Large language models (LLMs) have achieved impressive results across a range of natural language processing tasks, but their potential to generate harmful content has raised serious safety concerns. Current toxicity detectors primarily rely on single-label benchmarks, which cannot adequately capture the inherently ambiguous and multi-dimensional nature of real-world toxic prompts. This limitation results in biased evaluations, including missed toxic detections and false positives, undermining the reliability of existing detectors. Additionally, gathering comprehensive multi-label annotations across fine-grained toxicity categories is prohibitively costly, further hindering effective evaluation and development. To tackle these issues, we introduce three novel multi-label benchmarks for toxicity detection: \textbf{Q-A-MLL}, \textbf{R-A-MLL}, and \textbf{H-X-MLL}, derived from public toxicity datasets and annotated according to a detailed 15-category taxonomy. We further provide a theoretical proof that, on our released datasets, training with pseudo-labels yields better performance than directly learning from single-label supervision. In addition, we develop a pseudo-label-based toxicity detection method. Extensive experimental results show that our approach significantly surpasses advanced baselines, including GPT-4o and DeepSeek, thus enabling more accurate and reliable evaluation of multi-label toxicity in LLM-generated content.
Paper Structure (17 sections, 2 theorems, 9 equations, 11 figures, 12 tables)

This paper contains 17 sections, 2 theorems, 9 equations, 11 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Limitations of Existing LLM Toxicity Detection Benchmarks
  3. Three Multi-Labels LLM Toxicity Detection Benchmark with Human-Annotation
  4. Theoretical Analysis and the Proposed Method
  5. Using pseudo-labels yields better performance in PLMLL
  6. Proposed Method
  7. Experiments
  8. Conclusion and Limitation
  9. Broader Impact.
  10. related work
  11. Toxicity Detection In LLMs
  12. Learning with Partially Labeled Multi-Label Data
  13. Others
  14. Experimental Setup
  15. Backbone Models.
  16. ...and 2 more sections

Key Result

Proposition 4.1

Suppose an SPMLL pseudo-label-based method has an unreliability degree $\xi$, where $0 \leq \xi < 1$. Let $\theta = c \log \frac{2}{1+\xi}$, and suppose the Natarajan dimension The Natarajan dimension Natarajan1989 generalizes the VC-dimension to multi-class classification by measuring the largest s Then, when $n > n_0(\mathcal{H}, \epsilon, \delta, \xi)$, we have $\mathcal{R}(\hat{h}) < \epsilon$

Figures (11)

  • Figure 1: Illustration of the multi-label nature of toxic prompts in the Q-A dataset cheng2024soft.
  • Figure 2: Illustration of the multi-label nature of toxic prompts in the Q-A dataset cheng2024soft. (a) PCA of prompt embeddings colored by toxicity category shows significant semantic overlap, indicating harmful attributes are not mutually exclusive. (b) shows label counts on the Q-A test set after multi-label annotation, compared with the original single-label counts.
  • Figure 3: Illustration of the limitations of current datasets. (a) demonstrates a missed detection case. (b) highlights a correct model prediction that is mistakenly penalized as incorrect due to incomplete single-label annotations.
  • Figure 3: Performance before and after fine‐tuning with our pseudo‐labels (LEPL‐MLL). We report mean Average Precision ($\uparrow$) / Label Ranking Loss ($\downarrow$) on three datasets.
  • Figure 4: (a) and (c) show the label-co-occurrence matrices for Q-A-MLL and R-A-MLL; each entry gives the conditional probability that the column label appears when the row label is present. Darker colour indicates stronger co-occurrence. (b) and (d) plot the label count statistics of the two datasets.
  • ...and 6 more figures

Theorems & Definitions (3)

  • Proposition 4.1: Theorem 4.1 in pmlr-v202-liu23ar
  • Theorem 4.2
  • proof