Chinese Cyberbullying Detection: Dataset, Method, and Validation

TL;DR

This work targets the gap in Chinese cyberbullying research by introducing CHNCI, a large-scale incident-based dataset with 220,676 comments across 91 real-world incidents. It develops a two-phase annotation pipeline that first generates pseudo labels via an ensemble of explanation-based detectors—paraphraser-based, Chain-of-Thought, and multi-agent prompts—and then relies on native annotators to finalize labels, guided by defined cyberbullying incident criteria. CHNCI is evaluated for both cyberbullying language detection and incident prediction, showing that explanation-based methods and ensemble voting outperform single approaches, with strong performance from open-source LLMs in zero-shot settings. The dataset and code provide a practical benchmark for detection and incident forecasting in Chinese, enabling research into social amplification and temporal dynamics of cyberbullying.

Abstract

Existing cyberbullying detection benchmarks were organized by the polarity of speech, such as "offensive" and "non-offensive", which were essentially hate speech detection. However, in the real world, cyberbullying often attracted widespread social attention through incidents. To address this problem, we propose a novel annotation method to construct a cyberbullying dataset that organized by incidents. The constructed CHNCI is the first Chinese cyberbullying incident detection dataset, which consists of 220,676 comments in 91 incidents. Specifically, we first combine three cyberbullying detection methods based on explanations generation as an ensemble method to generate the pseudo labels, and then let human annotators judge these labels. Then we propose the evaluation criteria for validating whether it constitutes a cyberbullying incident. Experimental results demonstrate that the constructed dataset can be a benchmark for the tasks of cyberbullying detection and incident prediction. To the best of our knowledge, this is the first study for the Chinese cyberbullying incident detection task.

Figures (7)

• Figure 1: The overview of our method for building Chinese cyberbullying detection dataset organized by incidents. The data are collected from multiple mainstream Chinese social media platforms. Our method is composed of two phrases: machine-generated pseudo labels and manual annotation. The first phase combines three cyberbullying detection methods based on explanations as an ensemble method to generate the pseudo labels. The second phase utilizes native Chinese annotators to judge the pseudo labels with generated explanations.
• Figure 2: Screenshot of an annotation example on the annotation website. The red text indicates the English translation.
• Figure 3: Category distribution of the CHNCI dataset.
• Figure 4: Performance Comparison with Baseline Methods
• Figure 5: The process of dataset validation. Step 1: Scraping comments related to cyberbullying incidents, which may include offensive expressions such as "????"(Hulk Shandong), "????" (The judgment is too lenient), and "????" (A bunch of idiots). Step 2: Data Preparation. Step 3: Cyberbullying language detection. Step 4: Data statistics and Cyberbullying incidents prediction.