Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detection

Zewen Bai, Shengdi Yin, Junyu Lu, Jingjie Zeng, Haohao Zhu, Yuanyuan Sun, Liang Yang, Hongfei Lin

TL;DR

STATE ToxiCN delivers the first span-level, target-aware toxicity extraction dataset for Chinese hate speech, comprising 8,029 posts and 9,533 Target-Argument-Hateful-Group quadruples, plus an annotated hateful slang lexicon. The work evaluates a broad set of models, showing that fine-tuned models substantially outperform API-based LLMs on span boundary tasks and complex target-argument-hateful predictions, though challenges remain in precise extraction and slang understanding. Key contributions include the dataset, the interpretable slang lexicon, and a thorough comparison of span-level extraction across models, providing resources and benchmarks to advance Chinese hate-speech detection. The findings highlight the need for targeted fine-tuning and knowledge-rich language models to handle span-boundaries, target grouping, and culturally nuanced slang in real-world settings.

Abstract

The proliferation of hate speech has caused significant harm to society. The intensity and directionality of hate are closely tied to the target and argument it is associated with. However, research on hate speech detection in Chinese has lagged behind, and existing datasets lack span-level fine-grained annotations. Furthermore, the lack of research on Chinese hateful slang poses a significant challenge. In this paper, we provide a solution for fine-grained detection of Chinese hate speech. First, we construct a dataset containing Target-Argument-Hateful-Group quadruples (STATE ToxiCN), which is the first span-level Chinese hate speech dataset. Secondly, we evaluate the span-level hate speech detection performance of existing models using STATE ToxiCN. Finally, we conduct the first study on Chinese hateful slang and evaluate the ability of LLMs to detect such expressions. Our work contributes valuable resources and insights to advance span-level hate speech detection in Chinese.

STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detection

TL;DR

STATE ToxiCN delivers the first span-level, target-aware toxicity extraction dataset for Chinese hate speech, comprising 8,029 posts and 9,533 Target-Argument-Hateful-Group quadruples, plus an annotated hateful slang lexicon. The work evaluates a broad set of models, showing that fine-tuned models substantially outperform API-based LLMs on span boundary tasks and complex target-argument-hateful predictions, though challenges remain in precise extraction and slang understanding. Key contributions include the dataset, the interpretable slang lexicon, and a thorough comparison of span-level extraction across models, providing resources and benchmarks to advance Chinese hate-speech detection. The findings highlight the need for targeted fine-tuning and knowledge-rich language models to handle span-boundaries, target grouping, and culturally nuanced slang in real-world settings.

Abstract

The proliferation of hate speech has caused significant harm to society. The intensity and directionality of hate are closely tied to the target and argument it is associated with. However, research on hate speech detection in Chinese has lagged behind, and existing datasets lack span-level fine-grained annotations. Furthermore, the lack of research on Chinese hateful slang poses a significant challenge. In this paper, we provide a solution for fine-grained detection of Chinese hate speech. First, we construct a dataset containing Target-Argument-Hateful-Group quadruples (STATE ToxiCN), which is the first span-level Chinese hate speech dataset. Secondly, we evaluate the span-level hate speech detection performance of existing models using STATE ToxiCN. Finally, we conduct the first study on Chinese hateful slang and evaluate the ability of LLMs to detect such expressions. Our work contributes valuable resources and insights to advance span-level hate speech detection in Chinese.
Paper Structure (34 sections, 1 figure, 7 tables)

This paper contains 34 sections, 1 figure, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hate Speech Detection.
  4. Chinese Hate Speech Dataset.
  5. Dataset Construction
  6. Overview
  7. Data Source and Filtering
  8. Data Annotation
  9. Annotation Guidelines
  10. Mitigating Bias.
  11. Annotation Procedure
  12. Cross-validation and Expert Arbitration.
  13. Lexicon Annotation.
  14. Data Description
  15. Experiment
  16. ...and 19 more sections

Figures (1)

  • Figure 1: Examples of Chinese hateful slang understanding analysis with LLM. The hateful slang terms and texts appear in black, ShieldGemma-v3 explanations are in green, DeepSeek-v3 explanations are in red, and human annotator explanations are in blue.