Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When to Invoke: Refining LLM Fairness with Toxicity Assessment

Jing Ren, Bowen Li, Ziqi Xu, Renqiang Luo, Shuo Yu, Xin Ye, Haytham Fayek, Xiaodong Li, Feng Xia

TL;DR

FairToT addresses inconsistent LLM toxicity judgments across demographic groups, especially for implicit hate speech, by introducing an inference-time, prompt-based refinement framework. It uses two interpretable fairness indicators—Sentence Fairness Variance ($\theta^S$) and Entity Fairness Dispersion ($\theta^E$)—to detect cases where demographic variation is likely and conditionally invokes a three-stage mitigation that aligns entity-wide scores without changing model parameters. The evaluation on three implicit hate speech benchmarks shows substantial reductions in group-level disparities (SFV and EFD) with modest computational overhead. The work demonstrates that selective, inference-time refinement offers a practical path to fairer LLM-based moderation in real-world deployments.

Abstract

Large Language Models (LLMs) are increasingly used for toxicity assessment in online moderation systems, where fairness across demographic groups is essential for equitable treatment. However, LLMs often produce inconsistent toxicity judgements for subtle expressions, particularly those involving implicit hate speech, revealing underlying biases that are difficult to correct through standard training. This raises a key question that existing approaches often overlook: when should corrective mechanisms be invoked to ensure fair and reliable assessments? To address this, we propose FairToT, an inference-time framework that enhances LLM fairness through prompt-guided toxicity assessment. FairToT identifies cases where demographic-related variation is likely to occur and determines when additional assessment should be applied. In addition, we introduce two interpretable fairness indicators that detect such cases and improve inference consistency without modifying model parameters. Experiments on benchmark datasets show that FairToT reduces group-level disparities while maintaining stable and reliable toxicity predictions, demonstrating that inference-time refinement offers an effective and practical approach for fairness improvement in LLM-based toxicity assessment systems. The source code can be found at https://aisuko.github.io/fair-tot/.

When to Invoke: Refining LLM Fairness with Toxicity Assessment

TL;DR

FairToT addresses inconsistent LLM toxicity judgments across demographic groups, especially for implicit hate speech, by introducing an inference-time, prompt-based refinement framework. It uses two interpretable fairness indicators—Sentence Fairness Variance () and Entity Fairness Dispersion ()—to detect cases where demographic variation is likely and conditionally invokes a three-stage mitigation that aligns entity-wide scores without changing model parameters. The evaluation on three implicit hate speech benchmarks shows substantial reductions in group-level disparities (SFV and EFD) with modest computational overhead. The work demonstrates that selective, inference-time refinement offers a practical path to fairer LLM-based moderation in real-world deployments.

Abstract

Large Language Models (LLMs) are increasingly used for toxicity assessment in online moderation systems, where fairness across demographic groups is essential for equitable treatment. However, LLMs often produce inconsistent toxicity judgements for subtle expressions, particularly those involving implicit hate speech, revealing underlying biases that are difficult to correct through standard training. This raises a key question that existing approaches often overlook: when should corrective mechanisms be invoked to ensure fair and reliable assessments? To address this, we propose FairToT, an inference-time framework that enhances LLM fairness through prompt-guided toxicity assessment. FairToT identifies cases where demographic-related variation is likely to occur and determines when additional assessment should be applied. In addition, we introduce two interpretable fairness indicators that detect such cases and improve inference consistency without modifying model parameters. Experiments on benchmark datasets show that FairToT reduces group-level disparities while maintaining stable and reliable toxicity predictions, demonstrating that inference-time refinement offers an effective and practical approach for fairness improvement in LLM-based toxicity assessment systems. The source code can be found at https://aisuko.github.io/fair-tot/.
Paper Structure (31 sections, 10 equations, 4 figures, 4 tables)

This paper contains 31 sections, 10 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fairness in NLP
  4. LLM-based Web Moderation
  5. Methodology
  6. Bias Detection via Local-Global Fairness Quantification
  7. Sentence-level Bias Quantification
  8. Entity-level Bias Quantification
  9. Trigger Condition
  10. Combined Fairness Risk.
  11. Three-Stage Prompt-Based Mitigation
  12. Fairness Evaluation
  13. Sentence Fairness Variance (SFV)
  14. Entity Fairness Dispersion (EFD)
  15. Experimental Setup
  16. ...and 16 more sections

Figures (4)

  • Figure 1: Examples illustrating fair and biased outputs. A fair model should assign similar toxicity scores across demographic entities (e.g., 0.72, 0.71, 0.73) when the underlying sentiment is unchanged. In contrast, many models yield notably different scores (e.g., 0.88, 0.69, 0.78), revealing bias associated with specific demographic groups.
  • Figure 2: Overview of FairToT.
  • Figure 3: Visualisation of fairness improvements before and after applying FairToT using the GPT-3.5-Turbo backbone.
  • Figure 4: Token consumption and fairness gain analysis for the Three-Step Prompting (3SP) and In-Context Learning (ICL) components within the FairToT framework.