X-MuTeST: A Multilingual Benchmark for Explainable Hate Speech Detection and A Novel LLM-consulted Explanation Framework

TL;DR

X-MuTeST introduces a multilingual benchmark and a novel two-stage explainable hate speech detection framework that fuses human token-level rationales with an n-gram based explainability method and LLM-consulted explanations. The approach leverages language-specific encoders (Muril for Telugu/English and XLM-R for Hindi) and LLaMA-3.1 as the explanatory oracle, applying a two-stage training that first aligns attention with human rationales and then uses model-driven saliency targets. Evaluation across Hindi, Telugu, and English shows superior classification and explainability performance, with notable generalizability to HateXplain and HateBRXplain datasets, underscoring the method’s utility for under-resourced languages and multilingual settings. The work contributes benchmark rationales, a hybrid explainability mechanism, and a two-stage training paradigm that improves both trustworthiness and performance in multilingual hate speech detection. This has practical impact for social media moderation in linguistically diverse contexts and provides a resource for future research in explainable NLP for low-resource languages.

Abstract

Hate speech detection on social media faces challenges in both accuracy and explainability, especially for underexplored Indic languages. We propose a novel explainability-guided training framework, X-MuTeST (eXplainable Multilingual haTe Speech deTection), for hate speech detection that combines high-level semantic reasoning from large language models (LLMs) with traditional attention-enhancing techniques. We extend this research to Hindi and Telugu alongside English by providing benchmark human-annotated rationales for each word to justify the assigned class label. The X-MuTeST explainability method computes the difference between the prediction probabilities of the original text and those of unigrams, bigrams, and trigrams. Final explanations are computed as the union between LLM explanations and X-MuTeST explanations. We show that leveraging human rationales during training enhances both classification performance and explainability. Moreover, combining human rationales with our explainability method to refine the model attention yields further improvements. We evaluate explainability using Plausibility metrics such as Token-F1 and IOU-F1 and Faithfulness metrics such as Comprehensiveness and Sufficiency. By focusing on under-resourced languages, our work advances hate speech detection across diverse linguistic contexts. Our dataset includes token-level rationale annotations for 6,004 Hindi, 4,492 Telugu, and 6,334 English samples. Data and code are available on https://github.com/ziarehman30/X-MuTeST

Figures (5)

• Figure 1: Example of human and LLM rationales
• Figure 2: Iterative annotation procedure
• Figure 3: Architecture of LLM-consulted explanation framework
• Figure 4: Influence of $\alpha$ parameter on classification performance in stage-2 training
• Figure 5: Ablation on two stages of training