NLPineers@ NLU of Devanagari Script Languages 2025: Hate Speech Detection using Ensembling of BERT-based models

TL;DR

This work tackles hate speech detection in Devanagari-script languages (Hindi and Nepali) for CHIPSAL@COLING 2025 Subtask B by evaluating an ensemble of multilingual transformer models (XLM-RoBERTa, MuRIL, IndicBERT) and augmenting data via backtranslation. The authors demonstrate that a three-model ensemble (primary XLM-RoBERTa, secondary MuRIL, fallback MuRIL abusive) yields the best recall (0.7762) and competitive F1 (0.6914), with an overall accuracy of 0.8258 on the evaluation set. Data augmentation is carefully filtered by cosine similarity to preserve label integrity, and class imbalance is mitigated by duplicating hate speech examples. The study provides a foundation for hate speech detection in Devanagari-script languages and highlights the potential and limitations of multilingual BERT ensembles, paving the way for future improvements in robustness and scalability.

Abstract

This paper explores hate speech detection in Devanagari-scripted languages, focusing on Hindi and Nepali, for Subtask B of the CHIPSAL@COLING 2025 Shared Task. Using a range of transformer-based models such as XLM-RoBERTa, MURIL, and IndicBERT, we examine their effectiveness in navigating the nuanced boundary between hate speech and free expression. Our best performing model, implemented as ensemble of multilingual BERT models achieve Recall of 0.7762 (Rank 3/31 in terms of recall) and F1 score of 0.6914 (Rank 17/31). To address class imbalance, we used backtranslation for data augmentation, and cosine similarity to preserve label consistency after augmentation. This work emphasizes the need for hate speech detection in Devanagari-scripted languages and presents a foundation for further research.

Figures (2)

• Figure 1: Experiment Workflow.
• Figure 2: Examples from the dataset.