Take its Essence, Discard its Dross! Debiasing for Toxic Language Detection via Counterfactual Causal Effect

TL;DR

This work tackles lexical bias in toxic language detection by introducing the Counterfactual Causal Debiasing Framework (CCDF), which separates the useful, context-driven influence of biased tokens from their misleading direct impact. CCDF builds an ensemble feature from the original sentence and biased tokens, uses three branch models to capture both context and lexical bias, and employs counterfactual reasoning to remove the direct effect of bias on predictions while preserving indirect, context-empowered signals. Empirical results show CCDF achieves state-of-the-art accuracy and fairness on in-distribution data and substantially better generalization to out-of-distribution data, outperforming existing debiasing methods. The method provides a principled, causal approach to debiasing in TLD with potential applicability to other NLU tasks, while acknowledging limitations related to lexicon coverage and dialectal bias.

Abstract

Current methods of toxic language detection (TLD) typically rely on specific tokens to conduct decisions, which makes them suffer from lexical bias, leading to inferior performance and generalization. Lexical bias has both "useful" and "misleading" impacts on understanding toxicity. Unfortunately, instead of distinguishing between these impacts, current debiasing methods typically eliminate them indiscriminately, resulting in a degradation in the detection accuracy of the model. To this end, we propose a Counterfactual Causal Debiasing Framework (CCDF) to mitigate lexical bias in TLD. It preserves the "useful impact" of lexical bias and eliminates the "misleading impact". Specifically, we first represent the total effect of the original sentence and biased tokens on decisions from a causal view. We then conduct counterfactual inference to exclude the direct causal effect of lexical bias from the total effect. Empirical evaluations demonstrate that the debiased TLD model incorporating CCDF achieves state-of-the-art performance in both accuracy and fairness compared to competitive baselines applied on several vanilla models. The generalization capability of our model outperforms current debiased models for out-of-distribution data.

Figures (5)

• Figure 1: Due to the biased training, the TLD model is prone to identify all samples containing biased tokens, such as "n*gga", as toxic language. In this paper, we present a Counterfactual Causal Debiasing Framework to mitigate lexical bias by excluding the direct causal effect of biased tokens on model decisions from the total effect.
• Figure 2: Illustration of causal graph. (a) Factual scenario; (b, c) Counterfactual scenario. Where white nodes denote variables with observed values and gray nodes denote variables with counterfactual values
• Figure 3: The model diagram of CCDF, where Only X, Only B and Ensemble represent different branch models, i.e.$\mathcal{F}_E$, $\mathcal{F}_X$, and $\mathcal{F}_B$, respectively. The vector representations of the original sentence and biased tokens are obtained by the same encoder. $\mathcal{L}_f$, $\mathcal{L}_x$, $\mathcal{L}_e$, and $\mathcal{L}_b$ respectively refer to the loss values between each predicate logit (i.e.$Y_{e,x,b}$, $Y_x$, $Y_e$, and $Y_b$) and the ground-truth label.
• Figure 4: Comparison between (a) Factual TLD and (b) Counterfactual TLD using causal graph.
• Figure B1: Causal graph of ablated CCDF.