An Annotated Corpus of Arabic Tweets for Hate Speech Analysis
Wajdi Zaghouani, Md. Rafiul Biswas
TL;DR
The paper tackles the challenge of Arabic hate speech detection amid dialectal variation by constructing a large-scale, multilabel tweet corpus of 10,000 samples annotated for offensiveness and multiple hate-speech targets. It documents a robust data collection and annotation pipeline, including diverse Arab-country annotators and MicroMappers, and evaluates transformer baselines (notably AraBERTv2) for both offensive detection and target classification, achieving micro-F1s of 0.7865 and 0.6889, respectively. Inter-annotator agreement is substantial (IAA Fleiss' kappa = 0.8143), underscoring annotation reliability. The dataset and code are openly available under CC BY 4.0, providing a valuable resource to advance Arabic NLP hate-speech research and cross-dialect evaluation across sociolinguistic contexts.
Abstract
Identifying hate speech content in the Arabic language is challenging due to the rich quality of dialectal variations. This study introduces a multilabel hate speech dataset in the Arabic language. We have collected 10000 Arabic tweets and annotated each tweet, whether it contains offensive content or not. If a text contains offensive content, we further classify it into different hate speech targets such as religion, gender, politics, ethnicity, origin, and others. A text can contain either single or multiple targets. Multiple annotators are involved in the data annotation task. We calculated the inter-annotator agreement, which was reported to be 0.86 for offensive content and 0.71 for multiple hate speech targets. Finally, we evaluated the data annotation task by employing a different transformers-based model in which AraBERTv2 outperformed with a micro-F1 score of 0.7865 and an accuracy of 0.786.