Hierarchical Multi-Stage BERT Fusion Framework with Dual Attention for Enhanced Cyberbullying Detection in Social Media

TL;DR

This work addresses cyberbullying detection in social media by introducing a hierarchical multi-stage BERT fusion framework that integrates token- and sequence-level embeddings with auxiliary sentiment and topic features. It employs self-attention and cross-attention to align features and uses a hierarchical classification head with dynamic loss balancing to handle imbalanced data for multi-class and binary tasks. The approach demonstrates superior performance over strong baselines, achieving high accuracy, precision, recall, and F1, and offers a robust tool for moderating online content. Overall, the framework advances domain-specific NLP by fusing rich contextual representations with targeted auxiliary signals for improved detection and analysis of harmful content.

Abstract

Detecting and classifying cyberbullying in social media is hard because of the complex nature of online language and the changing nature of content. This study presents a multi-stage BERT fusion framework. It uses hierarchical embeddings, dual attention mechanisms, and extra features to improve detection of cyberbullying content. The framework combines BERT embeddings with features like sentiment and topic information. It uses self-attention and cross-attention to align features and has a hierarchical classification head for multi-category classification. A dynamic loss balancing strategy helps optimize learning and improves accuracy, precision, recall, and F1-score. These results show the model's strong performance and potential for broader use in analyzing social media content.

Figures (5)

• Figure 1: The multi-level BERT-fusion model.
• Figure 2: BERT embeddings, sentiment embeddings, and topic embeddings..
• Figure 3: Preprocessing pipeline for embedding tweets using GloVe and BERT.
• Figure 4: Model indicator change chart.
• Figure 5: Model indicator change chart.