Feature Selection Empowered BERT for Detection of Hate Speech with Vocabulary Augmentation

TL;DR

The paper addresses the challenge of detecting hate speech in evolving online language by proposing a data- and vocabulary-efficient fine-tuning approach for BERT. It combines TF-IDF-based selective sampling to reduce training data by about 25% with a lightweight vocabulary augmentation that adds domain-specific hate terms to the tokenizer. Empirical results on the Davidson dataset show that modest data reduction, coupled with targeted lexical expansion, preserves or improves performance while significantly cutting training time. This approach offers a scalable path for adaptive moderation in dynamic social media environments.

Abstract

Abusive speech on social media poses a persistent and evolving challenge, driven by the continuous emergence of novel slang and obfuscated terms designed to circumvent detection systems. In this work, we present a data efficient strategy for fine tuning BERT on hate speech classification by significantly reducing training set size without compromising performance. Our approach employs a TF IDF-based sample selection mechanism to retain only the most informative 75 percent of examples, thereby minimizing training overhead. To address the limitations of BERT's native vocabulary in capturing evolving hate speech terminology, we augment the tokenizer with domain-specific slang and lexical variants commonly found in abusive contexts. Experimental results on a widely used hate speech dataset demonstrate that our method achieves competitive performance while improving computational efficiency, highlighting its potential for scalable and adaptive abusive content moderation.

Figures (2)

• Figure 1: Architecture of our method
• Figure 2: Comparison of evaluation metrics (Accuracy, Precision, Recall, and F1 Score) for the vanilla model and various TF-IDF filtered variants (from 80% to 50%).