Creating and Evaluating Code-Mixed Nepali-English and Telugu-English Datasets for Abusive Language Detection Using Traditional and Deep Learning Models

TL;DR

This work introduces manually annotated Telugu-English and Nepali-English code-mixed datasets for abusive language detection, addressing a gap in low-resource multilingual NLP. It conducts a comprehensive evaluation across traditional ML, deep learning, and large language models using 10-fold cross-validation, highlighting the superior performance of transformer-based LLMs in cross-lingual, transliteration-rich settings. The study provides detailed preprocessing pipelines, dataset statistics, and code-mixing analyses (e.g., Code-Mixing Index) to support reproducibility and benchmark establishment. Its findings have practical implications for multilingual moderation on social platforms and establish benchmarks to guide future research in code-mixed abuse detection for underrepresented languages.

Abstract

With the growing presence of multilingual users on social media, detecting abusive language in code-mixed text has become increasingly challenging. Code-mixed communication, where users seamlessly switch between English and their native languages, poses difficulties for traditional abuse detection models, as offensive content may be context-dependent or obscured by linguistic blending. While abusive language detection has been extensively explored for high-resource languages like English and Hindi, low-resource languages such as Telugu and Nepali remain underrepresented, leaving gaps in effective moderation. In this study, we introduce a novel, manually annotated dataset of 2 thousand Telugu-English and 5 Nepali-English code-mixed comments, categorized as abusive and non-abusive, collected from various social media platforms. The dataset undergoes rigorous preprocessing before being evaluated across multiple Machine Learning (ML), Deep Learning (DL), and Large Language Models (LLMs). We experimented with models including Logistic Regression, Random Forest, Support Vector Machines (SVM), Neural Networks (NN), LSTM, CNN, and LLMs, optimizing their performance through hyperparameter tuning, and evaluate it using 10-fold cross-validation and statistical significance testing (t-test). Our findings provide key insights into the challenges of detecting abusive language in code-mixed settings and offer a comparative analysis of computational approaches. This study contributes to advancing NLP for low-resource languages by establishing benchmarks for abusive language detection in Telugu-English and Nepali-English code-mixed text. The dataset and insights can aid in the development of more robust moderation strategies for multilingual social media environments.

Figures (17)

• Figure 1: Distribution of Code-Mixing Index (CMI) for Nepali-English dataset.
• Figure 2: Distribution of Code-Mixing Index (CMI) for Telugu-English dataset.
• Figure 3: Workflow of Experimental setup
• Figure 4: F1-Score Trends for Logistic Regression
• Figure 5: F1-Score Trends for SVM