Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Harnessing Artificial Intelligence to Combat Online Hate: Exploring the Challenges and Opportunities of Large Language Models in Hate Speech Detection

Tharindu Kumarage, Amrita Bhattacharjee, Joshua Garland

TL;DR

This work evaluates large language models (LLMs) as hate speech detectors, combining a literature review with an empirical study on HateCheck using open-source and proprietary models (Llama-2, Falcon, GPT-3.5). It demonstrates that GPT-3.5 and Llama-2 achieve strong zero-shot performance (roughly 80–90% accuracy) while identifying key weaknesses such as target-specific errors and reliance on spurious cues in some models. The study further shows that simple, direct prompts often outperform more complex prompting strategies like context or chain-of-thought prompts in hate speech detection. The findings offer practical guidance on model choice, prompt design, and evaluation practices to improve reliability and fairness in automated moderation, while highlighting ongoing challenges in robustness and counter-speech contexts.

Abstract

Large language models (LLMs) excel in many diverse applications beyond language generation, e.g., translation, summarization, and sentiment analysis. One intriguing application is in text classification. This becomes pertinent in the realm of identifying hateful or toxic speech -- a domain fraught with challenges and ethical dilemmas. In our study, we have two objectives: firstly, to offer a literature review revolving around LLMs as classifiers, emphasizing their role in detecting and classifying hateful or toxic content. Subsequently, we explore the efficacy of several LLMs in classifying hate speech: identifying which LLMs excel in this task as well as their underlying attributes and training. Providing insight into the factors that contribute to an LLM proficiency (or lack thereof) in discerning hateful content. By combining a comprehensive literature review with an empirical analysis, our paper strives to shed light on the capabilities and constraints of LLMs in the crucial domain of hate speech detection.

Harnessing Artificial Intelligence to Combat Online Hate: Exploring the Challenges and Opportunities of Large Language Models in Hate Speech Detection

TL;DR

This work evaluates large language models (LLMs) as hate speech detectors, combining a literature review with an empirical study on HateCheck using open-source and proprietary models (Llama-2, Falcon, GPT-3.5). It demonstrates that GPT-3.5 and Llama-2 achieve strong zero-shot performance (roughly 80–90% accuracy) while identifying key weaknesses such as target-specific errors and reliance on spurious cues in some models. The study further shows that simple, direct prompts often outperform more complex prompting strategies like context or chain-of-thought prompts in hate speech detection. The findings offer practical guidance on model choice, prompt design, and evaluation practices to improve reliability and fairness in automated moderation, while highlighting ongoing challenges in robustness and counter-speech contexts.

Abstract

Large language models (LLMs) excel in many diverse applications beyond language generation, e.g., translation, summarization, and sentiment analysis. One intriguing application is in text classification. This becomes pertinent in the realm of identifying hateful or toxic speech -- a domain fraught with challenges and ethical dilemmas. In our study, we have two objectives: firstly, to offer a literature review revolving around LLMs as classifiers, emphasizing their role in detecting and classifying hateful or toxic content. Subsequently, we explore the efficacy of several LLMs in classifying hate speech: identifying which LLMs excel in this task as well as their underlying attributes and training. Providing insight into the factors that contribute to an LLM proficiency (or lack thereof) in discerning hateful content. By combining a comprehensive literature review with an empirical analysis, our paper strives to shed light on the capabilities and constraints of LLMs in the crucial domain of hate speech detection.
Paper Structure (17 sections, 3 figures, 3 tables)

This paper contains 17 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. LLMs as Text Classifiers or Annotators
  3. General Text Classifier or Annotator
  4. Hate Speech Classifiers
  5. Empirical Analysis
  6. Experiment Design
  7. LLM Selection
  8. Dataset Selection
  9. Hate Speech Detection
  10. Experiment Results
  11. RQ1: LLM's Hate Classification Performance
  12. RQ2: Effect of Prompting
  13. Discussion
  14. Answering RQ1: LLM's Robustness in Classifying Hate Speech
  15. Answering RQ2: Influence of Prompting Techniques
  16. ...and 2 more sections

Figures (3)

  • Figure 1: HateCheck dataset statistics
  • Figure 2: Error analysis on non-hate class
  • Figure 3: Prompt templates used for hate speech classification