Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection

Arij Riabi, Virginie Mouilleron, Menel Mahamdi, Wissam Antoun, Djamé Seddah

TL;DR

This work introduces Counter, a multilingual, pseudo-anonymized dataset for online radical content detection across English, French, and Arabic, accompanied by rich annotation schemes and synthetic data for bias analysis. It systematically analyzes how human label variation and sociodemographic factors influence model performance, and evaluates the role of annotation aggregation, multi-task learning, and out-of-domain NER signals. The paper demonstrates substantial cross-language variability in performance and fairness, highlighting biases tied to political views, ethnicity, and language register, and shows that synthetic data can probe these biases though with limitations. Overall, the study underscores the importance of transparency, careful annotation practices, and bias-aware approaches when building robust, fair radical-content detectors in multilingual settings.

Abstract

The proliferation of radical content on online platforms poses significant risks, including inciting violence and spreading extremist ideologies. Despite ongoing research, existing datasets and models often fail to address the complexities of multilingual and diverse data. To bridge this gap, we introduce a publicly available multilingual dataset annotated with radicalization levels, calls for action, and named entities in English, French, and Arabic. This dataset is pseudonymized to protect individual privacy while preserving contextual information. Beyond presenting our freely available dataset, we analyze the annotation process, highlighting biases and disagreements among annotators and their implications for model performance. Additionally, we use synthetic data to investigate the influence of socio-demographic traits on annotation patterns and model predictions. Our work offers a comprehensive examination of the challenges and opportunities in building robust datasets for radical content detection, emphasizing the importance of fairness and transparency in model development.

Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection

TL;DR

This work introduces Counter, a multilingual, pseudo-anonymized dataset for online radical content detection across English, French, and Arabic, accompanied by rich annotation schemes and synthetic data for bias analysis. It systematically analyzes how human label variation and sociodemographic factors influence model performance, and evaluates the role of annotation aggregation, multi-task learning, and out-of-domain NER signals. The paper demonstrates substantial cross-language variability in performance and fairness, highlighting biases tied to political views, ethnicity, and language register, and shows that synthetic data can probe these biases though with limitations. Overall, the study underscores the importance of transparency, careful annotation practices, and bias-aware approaches when building robust, fair radical-content detectors in multilingual settings.

Abstract

The proliferation of radical content on online platforms poses significant risks, including inciting violence and spreading extremist ideologies. Despite ongoing research, existing datasets and models often fail to address the complexities of multilingual and diverse data. To bridge this gap, we introduce a publicly available multilingual dataset annotated with radicalization levels, calls for action, and named entities in English, French, and Arabic. This dataset is pseudonymized to protect individual privacy while preserving contextual information. Beyond presenting our freely available dataset, we analyze the annotation process, highlighting biases and disagreements among annotators and their implications for model performance. Additionally, we use synthetic data to investigate the influence of socio-demographic traits on annotation patterns and model predictions. Our work offers a comprehensive examination of the challenges and opportunities in building robust datasets for radical content detection, emphasizing the importance of fairness and transparency in model development.

Paper Structure

This paper contains 37 sections, 6 equations, 21 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Online Radical Content Detection
  3. NLP for radical content detection
  4. Investigating radicalized users
  5. Detecting radical content
  6. Counter: Radical Content Dataset
  7. Data collection
  8. Prescriptive Annotation
  9. Descriptive Annotation
  10. Pseudononymization and NER annotations
  11. Synthetic Data for Bias Analysis
  12. Results
  13. Discussion
  14. How about Human Label Variations?
  15. Bias Analysis: What can we infer?
  16. ...and 22 more sections

Figures (21)

  • Figure 1: Average Macro-F1 variations for the attributes for XLM-T model for the synthetic English set.
  • Figure 3: Data source distributions for English, French, and Arabic
  • Figure 4: Data length distributions for English, French, and Arabic
  • Figure 5: Data distributions for English, French, and Arabic
  • Figure 6: Examples from the synthetic data
  • ...and 16 more figures