On the Role of Speech Data in Reducing Toxicity Detection Bias

TL;DR

This work investigates whether speech data can mitigate biases in toxicity detection by comparing speech-based classifiers with text-based cascaded systems. It introduces a high-quality, multilingual MuTOX annotation protocol for English and Spanish, enabling a fairness audit of group mentions in toxicity judgments. The study finds that incorporating speech at inference reduces false positives for group mentions and improves performance on ambiguous samples, though speech-based models can still exhibit group-bias biases. It concludes that multimodal approaches offer meaningful benefits for toxicity detection, while transcription quality alone does not resolve bias, and provides practical recommendations for dataset construction and deployment in real-world, multimodal contexts. All findings highlight the importance of careful annotation, multimodality, and domain-aware evaluation for fair toxicity detection systems.

Abstract

Text toxicity detection systems exhibit significant biases, producing disproportionate rates of false positives on samples mentioning demographic groups. But what about toxicity detection in speech? To investigate the extent to which text-based biases are mitigated by speech-based systems, we produce a set of high-quality group annotations for the multilingual MuTox dataset, and then leverage these annotations to systematically compare speech- and text-based toxicity classifiers. Our findings indicate that access to speech data during inference supports reduced bias against group mentions, particularly for ambiguous and disagreement-inducing samples. Our results also suggest that improving classifiers, rather than transcription pipelines, is more helpful for reducing group bias. We publicly release our annotations and provide recommendations for future toxicity dataset construction.

Figures (8)

• Figure 1: (a) Number of samples marked as toxic ("Yes"), not toxic ("No"), impossible to decide ("Cannot say"), or where annotators could not reach consensus ("No consensus") for English and Spanish. (b) Number of samples marked as mentioning or referring to a group. (c) Percentage of samples per group marked as toxic.
• Figure 2: (a) F-score, (b) precision, and (c) recall of each classifier, for samples with and without group mentions. etox and detoxify show lower $F_1$-score when a group is mentioned, whereas MuTOX-ASR and MuTOX show a slight increase. MuTOX is the only classifier to increase both precision and recall when groups are mentioned.
• Figure 3: (a) Classifier false positive rate (FPR) for samples with and without group mentions. (b) FPR of each classifier on samples annotators marked as "Cannot say" or "No consensus." (c) FPR on ambiguous samples with and without group mentions. detoxify and MuTOX have an FPR of zero on ambiguous samples, while both etox and MuTOX-ASR demonstrate increased FPR when ambiguous samples mention groups.
• Figure 4: False positive rate (FPR) of MuTOX and MuTOX-ASR on samples mentioning specific (a) gender identities, (b) racial or ethnic groups, (c) religious groups. (a) Mutox ASR shows a higher FPR for samples mentioning women than for other samples, whereas MuTOX's FPR decreases. (b) MuTOX-ASR shows a stronger bias against samples mentioning either White or Black people when compared to MuTOX. (c) Similarly, MuTOX-ASR shows a stronger bias against religious group mentions than MuTOX.
• Figure 5: (a) $F_1$-score of cascaded ASR-based classifiers with original ASR transcripts and annotator-corrected transcripts. (b) FPR on samples mentioning groups. Corrected transcripts only marginally improve model performance but have little to no impact on FPR.