Bridging Gaps in Hate Speech Detection: Meta-Collections and Benchmarks for Low-Resource Iberian Languages

TL;DR

This work tackles the scarcity of hate speech resources for underrepresented Iberian languages by building MetaHateES, a unified meta-collection of European Spanish datasets with harmonized labels and metadata. It extends the resource by generating European Portuguese and two Galician variants (Spanish-like and Portuguese-like) to create aligned multilingual corpora, enabling cross-lingual benchmarking. The study benchmarks state-of-the-art LLMs (e.g., Llama 3.1, Nemo) and a multilingual encoder (BERT) in zero-shot, few-shot, and fine-tuning settings, and conducts cross-lingual evaluations to assess transfer across Iberian languages. Key findings show that fine-tuning with language-specific data yields the best performance, cross-lingual transfer is strongest among closely related languages, and Galician’s internal variation significantly impacts model performance, underscoring the need for variety-aware multilingual approaches. The framework provides a foundation for equitable hate speech benchmarking in underrepresented European languages and guides future work toward broader language coverage and mitigation of translation biases.

Abstract

Hate speech poses a serious threat to social cohesion and individual well-being, particularly on social media, where it spreads rapidly. While research on hate speech detection has progressed, it remains largely focused on English, resulting in limited resources and benchmarks for low-resource languages. Moreover, many of these languages have multiple linguistic varieties, a factor often overlooked in current approaches. At the same time, large language models require substantial amounts of data to perform reliably, a requirement that low-resource languages often cannot meet. In this work, we address these gaps by compiling a meta-collection of hate speech datasets for European Spanish, standardised with unified labels and metadata. This collection is based on a systematic analysis and integration of existing resources, aiming to bridge the data gap and support more consistent and scalable hate speech detection. We extended this collection by translating it into European Portuguese and into a Galician standard that is more convergent with Spanish and another Galician variant that is more convergent with Portuguese, creating aligned multilingual corpora. Using these resources, we establish new benchmarks for hate speech detection in Iberian languages. We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, providing baseline results for future research. Moreover, we perform a cross-lingual analysis with our target languages. Our findings underscore the importance of multilingual and variety-aware approaches in hate speech detection and offer a foundation for improved benchmarking in underrepresented European languages.

Figures (5)

• Figure 1: Word clouds of topics for Spanish, Portuguese, Galician (ES) and Galician (PT).
• Figure 2: t-SNE visualisation of topics for Spanish, Portuguese, Galician (ES) and Galician (PT).
• Figure 3: Radar plot showing the percentage of posts that contain a word associated with the Plutchik emotions for hate and non-hate data, for Spanish, Portuguese, Galician (ES) and Galician (PT).
• Figure 4: For each language, Spanish, Portuguese, Galician (ES) and Galician (PT), distribution of verb tenses in hate and non-hate posts (left). And distribution of pronouns in hate and non-hate speech posts (right).
• Figure 5: Cross-lingual evaluation of BERT-based classifiers fine-tuned on each training language (rows) and evaluated on each target language (columns). Values represent F1 scores (macro-averaged) for the binary classification task of hate speech detection.