Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HMS-BERT: Hybrid Multi-Task Self-Training for Multilingual and Multi-Label Cyberbullying Detection

Zixin Feng, Xinying Cui, Yifan Sun, Zheng Wei, Jiachen Yuan, Jiazhen Hu, Ning Xin, Md Maruf Hasan

Abstract

Cyberbullying on social media is inherently multilingual and multi-faceted, where abusive behaviors often overlap across multiple categories. Existing methods are commonly limited by monolingual assumptions or single-task formulations, which restrict their effectiveness in realistic multilingual and multi-label scenarios. In this paper, we propose HMS-BERT, a hybrid multi-task self-training framework for multilingual and multi-label cyberbullying detection. Built upon a pretrained multilingual BERT backbone, HMS-BERT integrates contextual representations with handcrafted linguistic features and jointly optimizes a fine-grained multi-label abuse classification task and a three-class main classification task. To address labeled data scarcity in low-resource languages, an iterative self-training strategy with confidence-based pseudo-labeling is introduced to facilitate cross-lingual knowledge transfer. Experiments on four public datasets demonstrate that HMS-BERT achieves strong performance, attaining a macro F1-score of up to 0.9847 on the multi-label task and an accuracy of 0.6775 on the main classification task. Ablation studies further verify the effectiveness of the proposed components.

HMS-BERT: Hybrid Multi-Task Self-Training for Multilingual and Multi-Label Cyberbullying Detection

Abstract

Cyberbullying on social media is inherently multilingual and multi-faceted, where abusive behaviors often overlap across multiple categories. Existing methods are commonly limited by monolingual assumptions or single-task formulations, which restrict their effectiveness in realistic multilingual and multi-label scenarios. In this paper, we propose HMS-BERT, a hybrid multi-task self-training framework for multilingual and multi-label cyberbullying detection. Built upon a pretrained multilingual BERT backbone, HMS-BERT integrates contextual representations with handcrafted linguistic features and jointly optimizes a fine-grained multi-label abuse classification task and a three-class main classification task. To address labeled data scarcity in low-resource languages, an iterative self-training strategy with confidence-based pseudo-labeling is introduced to facilitate cross-lingual knowledge transfer. Experiments on four public datasets demonstrate that HMS-BERT achieves strong performance, attaining a macro F1-score of up to 0.9847 on the multi-label task and an accuracy of 0.6775 on the main classification task. Ablation studies further verify the effectiveness of the proposed components.
Paper Structure (48 sections, 9 equations, 5 figures, 5 tables, 1 algorithm)

This paper contains 48 sections, 9 equations, 5 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Literature Review
  4. Introduction: Machine Learning in Cyberbullying Detection
  5. The Rise of Pre-trained Language Models (PLMs)
  6. Methodological Progress
  7. Multi-label Detection
  8. The Multi-faceted Nature of Cyberbullying
  9. Data Scarcity in Low-Resource Settings
  10. Research Gap
  11. Methodology
  12. Data Preprocessing
  13. Workflow
  14. Input Representation and Feature Fusion
  15. Contextual Embedding with mBERT
  16. ...and 33 more sections

Figures (5)

  • Figure 1: Flowchart of Data Preprocessing
  • Figure 2: The workflow of the multilingual input representation, semantic encoding, feature enhancement, and multi-label classification.
  • Figure 3: Step2-Step3: Pre-trained network to Fine-tuned network
  • Figure 4: Cyberbullying Classification Statistics
  • Figure 5: Word cloud visualization of all data, illustrating the lexical features of cyberbullying content across languages.