Generative AI may backfire for counterspeech

TL;DR

It is found that non-contextualized counterspeech employing a warning-of-consequence strategy significantly reduces online hate speech.

Abstract

Online hate speech poses a serious threat to individual well-being and societal cohesion. A promising solution to curb online hate speech is counterspeech. Counterspeech is aimed at encouraging users to reconsider hateful posts by direct replies. However, current methods lack scalability due to the need for human intervention or fail to adapt to the specific context of the post. A potential remedy is the use of generative AI, specifically large language models (LLMs), to write tailored counterspeech messages. In this paper, we analyze whether contextualized counterspeech generated by state-of-the-art LLMs is effective in curbing online hate speech. To do so, we conducted a large-scale, pre-registered field experiment (N=2,664) on the social media platform Twitter/X. Our experiment followed a 2x2 between-subjects design and, additionally, a control condition with no counterspeech. On the one hand, users posting hateful content on Twitter/X were randomly assigned to receive either (a) contextualized counterspeech or (b) non-contextualized counterspeech. Here, the former is generated through LLMs, while the latter relies on predefined, generic messages. On the other hand, we tested two counterspeech strategies: (a) promoting empathy and (b) warning about the consequences of online misbehavior. We then measured whether users deleted their initial hateful posts and whether their behavior changed after the counterspeech intervention (e.g., whether users adopted a less toxic language). We find that non-contextualized counterspeech employing a warning-of-consequence strategy significantly reduces online hate speech. However, contextualized counterspeech generated by LLMs proves ineffective and may even backfire.

Figures (5)

• Figure 1: Overview of our field experiment.
• Figure 2: Average (a) rate of deleted posts, (b) number of hateful posts after the intervention, and (c) relative change in toxicity and standard errors (bars) by experimental condition. [$\uparrow$] ([$\downarrow$]) indicates that a [positive] ([negative]) outcome is associated with an [increase] ([decrease]) in the outcome values.
• Figure 3: Treatment effect of an intervention relative to the control condition (=no counterspeech intervention) for (a) Rate of deleted posts, (b) Number of hateful posts, and (c) Relative change in toxicity. Shown are the estimated coefficients from our linear regression model (symbol) as well as 95 % (thin), and 90 % (thick) confidence intervals. [$\uparrow$] ([$\downarrow$]) indicates that a [positive] ([negative]) outcome is associated with an [increase] ([decrease]) in the outcome values.
• Figure 4: Treatment effect of contextualized vs. non-contextualized counterspeech for (a) Rate of deleted posts, (b) Number of hateful posts, and (c) Relative change in toxicity. Shown are the estimated coefficients from our linear regression model (dot) measuring the relative effect of generic ( Non-contextualized) vs. contextualized ( Contextualized) counterspeech for the respective strategy as well as 95 % (thin), and 90 % (thick) confidence intervals. [$\uparrow$] ([$\downarrow$]) indicates that a [positive] ([negative]) outcome is associated with an [increase] ([decrease]) in the outcome values.
• Figure 5: Example of one of our human-controlled accounts.