Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Names Change Verdicts: Intervention Consistency Reveals Systematic Bias in LLM Decision-Making

Abhinaba Basu, Pavan Chakraborty

Abstract

Large language models (LLMs) are increasingly used for high-stakes decisions, yet their susceptibility to spurious features remains poorly characterized. We introduce ICE-Guard, a framework applying intervention consistency testing to detect three types of spurious feature reliance: demographic (name/race swaps), authority (credential/prestige swaps), and framing (positive/negative restatements). Across 3,000 vignettes spanning 10 high-stakes domains, we evaluate 11 LLMs from 8 families and find that (1) authority bias (mean 5.8%) and framing bias (5.0%) substantially exceed demographic bias (2.2%), challenging the field's narrow focus on demographics; (2) bias concentrates in specific domains -- finance shows 22.6% authority bias while criminal justice shows only 2.8%; (3) structured decomposition, where the LLM extracts features and a deterministic rubric decides, reduces flip rates by up to 100% (median 49% across 9 models). We demonstrate an ICE-guided detect-diagnose-mitigate-verify loop achieving cumulative 78% bias reduction via iterative prompt patching. Validation against real COMPAS recidivism data shows COMPAS-derived flip rates exceed pooled synthetic rates, suggesting our benchmark provides a conservative estimate of real-world bias. Code and data are publicly available.

When Names Change Verdicts: Intervention Consistency Reveals Systematic Bias in LLM Decision-Making

Abstract

Large language models (LLMs) are increasingly used for high-stakes decisions, yet their susceptibility to spurious features remains poorly characterized. We introduce ICE-Guard, a framework applying intervention consistency testing to detect three types of spurious feature reliance: demographic (name/race swaps), authority (credential/prestige swaps), and framing (positive/negative restatements). Across 3,000 vignettes spanning 10 high-stakes domains, we evaluate 11 LLMs from 8 families and find that (1) authority bias (mean 5.8%) and framing bias (5.0%) substantially exceed demographic bias (2.2%), challenging the field's narrow focus on demographics; (2) bias concentrates in specific domains -- finance shows 22.6% authority bias while criminal justice shows only 2.8%; (3) structured decomposition, where the LLM extracts features and a deterministic rubric decides, reduces flip rates by up to 100% (median 49% across 9 models). We demonstrate an ICE-guided detect-diagnose-mitigate-verify loop achieving cumulative 78% bias reduction via iterative prompt patching. Validation against real COMPAS recidivism data shows COMPAS-derived flip rates exceed pooled synthetic rates, suggesting our benchmark provides a conservative estimate of real-world bias. Code and data are publicly available.
Paper Structure (52 sections, 4 equations, 2 figures, 8 tables)

This paper contains 52 sections, 4 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. LLM Bias Benchmarks.
  5. Fairness in Decision Systems.
  6. Counterfactual Fairness.
  7. Intervention Consistency.
  8. LLM Self-Evaluation.
  9. ICE-Guard Framework
  10. Intervention Consistency for Decisions
  11. Invariance Principle.
  12. Flip Rate.
  13. Three Bias Intervention Types
  14. Demographic Intervention ($\mathcal{O}_\text{dem}$).
  15. Authority Intervention ($\mathcal{O}_\text{auth}$).
  16. ...and 37 more sections

Figures (2)

  • Figure 1: Flip rates (%) by domain and bias type, pooled across 11 models. The dashed line marks the 5% noise baseline. Finance is the most vulnerable domain (22.6% authority, 20.4% framing). Authority and framing bias consistently exceed demographic bias.
  • Figure 2: Free-form vs. structured flip rates with 95% Wilson CIs. Percentages show relative reduction. Structured decomposition reduces bias for 7 of 8 plotted models (GPT-5.2, which shows $+$68% increase, is in Table \ref{['tab:structured_full']}).