ViTHSD: Exploiting Hatred by Targets for Hate Speech Detection on Vietnamese Social Media Texts

TL;DR

This work addresses real-time, interpretable hate speech detection on Vietnamese social media by introducing ViTHSD, a 10k-comment dataset with five targets and four hate levels. It proposes a baseline architecture that combines BERT-based representations with Bi-GRU-LSTM-CNN and develops a streaming pipeline using Kafka and Spark to process live comments. Empirical results show strong target-only performance from XLM-R-based architectures (Macro F1 ≈ 72%) and competitive target+level performance from ViSoBERT-based approaches, with ViSoBERT offering superior streaming efficiency. The study demonstrates practical viability for real-time, targeted hate speech moderation and outlines future enhancements, including lexical normalization and the use of large language models for Vietnamese hate speech understanding.

Abstract

The growth of social networks makes toxic content spread rapidly. Hate speech detection is a task to help decrease the number of harmful comments. With the diversity in the hate speech created by users, it is necessary to interpret the hate speech besides detecting it. Hence, we propose a methodology to construct a system for targeted hate speech detection from online streaming texts from social media. We first introduce the ViTHSD - a targeted hate speech detection dataset for Vietnamese Social Media Texts. The dataset contains 10K comments, each comment is labeled to specific targets with three levels: clean, offensive, and hate. There are 5 targets in the dataset, and each target is labeled with the corresponding level manually by humans with strict annotation guidelines. The inter-annotator agreement obtained from the dataset is 0.45 by Cohen's Kappa index, which is indicated as a moderate level. Then, we construct a baseline for this task by combining the Bi-GRU-LSTM-CNN with the pre-trained language model to leverage the power of text representation of BERTology. Finally, we suggest a methodology to integrate the baseline model for targeted hate speech detection into the online streaming system for practical application in preventing hateful and offensive content on social media.

Figures (14)

• Figure 1: An example of annotated data by users on Google sheet.
• Figure 2: Data annotation process for the ViTHSD dataset with $k$ > 0.4 is acceptable landis1977measurement.
• Figure 3: The confusion matrix between the annotators in a set of 5,000 comments in each target without levels, computed by the Cohen kappa index ($k$)
• Figure 4: The Cohen's kappa index ($k$) was used to compute the confusion matrix between annotators in a set of 10,000 comments for each target without levels.
• Figure 5: The Cohen's kappa index ($k$) was used to calculate the confusion matrix between the annotators on 5,000 comments per target.