SoftHateBench: Evaluating Moderation Models Against Reasoning-Driven, Policy-Compliant Hostility

TL;DR

SoftHateBench addresses a gap in moderation by focusing on reasoning-driven, policy-compliant hostility that evades surface-level detectors. The authors combine the Argumentum Model of Topics (AMT) with Relevance Theory (RT) to build a reverse-generation, RT-guided beam search that constructs surface text from an explicit argumentative chain while preserving the hostile stance. Empirical results show a substantial drop in detection performance across encoders, LLMs, and safety models when facing soft hate, though introducing AMT intermediates (P and M) improves robustness, underscoring the need for reasoning-aware moderation. The benchmark spans 7 sociocultural domains and 28 target groups with 4,745 core soft-hate instances and 14,235 soft variants, enabling rigorous evaluation and guiding development of more resilient moderation systems.

Abstract

Online hate on social media ranges from overt slurs and threats (\emph{hard hate speech}) to \emph{soft hate speech}: discourse that appears reasonable on the surface but uses framing and value-based arguments to steer audiences toward blaming or excluding a target group. We hypothesize that current moderation systems, largely optimized for surface toxicity cues, are not robust to this reasoning-driven hostility, yet existing benchmarks do not measure this gap systematically. We introduce \textbf{\textsc{SoftHateBench}}, a generative benchmark that produces soft-hate variants while preserving the underlying hostile standpoint. To generate soft hate, we integrate the \emph{Argumentum Model of Topics} (AMT) and \emph{Relevance Theory} (RT) in a unified framework: AMT provides the backbone argument structure for rewriting an explicit hateful standpoint into a seemingly neutral discussion while preserving the stance, and RT guides generation to keep the AMT chain logically coherent. The benchmark spans \textbf{7} sociocultural domains and \textbf{28} target groups, comprising \textbf{4,745} soft-hate instances. Evaluations across encoder-based detectors, general-purpose LLMs, and safety models show a consistent drop from hard to soft tiers: systems that detect explicit hostility often fail when the same stance is conveyed through subtle, reasoning-based language. \textcolor{red}{\textbf{Disclaimer.} Contains offensive examples used solely for research.}

Figures (6)

• Figure 1: Hard vs. soft hate speech and why moderation differs. Top: explicit/implicit hard hate uses overt insults or thinly veiled metaphors, which both audiences and filters recognize as hostile, leading to rejection. Bottom: soft hate uses neutral-sounding claims and value-based reasoning (e.g., safety) to steer readers toward a discriminatory conclusion, often passing screening and being accepted.
• Figure 2: AMT inference and our reverse-generation procedure. Left: Canonical AMT chain: a material step fuses the Endoxon ($E$) and Datum ($D$) into a Premise ($P$); a procedural step applies a Maxim ($M$) under a Locus ($L$) to justify the Standpoint ($S$). Right: Reverse AMT generation used in SoftHateBench: starting from $(S,L)$, instantiate $M$ to derive $P$, then decompose $P$ into $(E,D)$. Only $(E,D)$ are presented in the final text, while $(P,L,M,S)$ remain latent.
• Figure 3: Effect–Cost quadrant for the same standpoint, illustrating how premises trade off support (Effect) and effort (Cost) and why our search favors the high-Effect/low-Cost.
• Figure 4: Target-group distribution in SoftHateBench. Colors denote Level 1 hate-speech domains; bars show Level 2 subgroups (referred to as target groups). Values above bars indicate sample counts. Abbreviations: African Am. = African American, Bloc = Regional Bloc, MENA = Middle East and North African, S. Asian = South/Southeast Asian.
• Figure 5: Domain-wise performance on SoftHateBench. Each radar chart shows a single model’s HSR (%) across Level 1 domains: Race, Religion, Sexual Orientation, Nationality/Region, Gender, Socio-economic Class, and Politics/Ideology; “All” is the macro average over domains. Curves compare Hard (green) against the three soft tiers—$\text{Soft}_{\text{base}}$ (orange), $\text{Soft}_{\text{GV}}$ (blue), and $\text{Soft}_{\text{HV}}$ (magenta). Larger shaded area indicates higher HSR; axes span 0–100.