Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DIF: A Framework for Benchmarking and Verifying Implicit Bias in LLMs

Lake Yin, Fan Huang

TL;DR

This paper tackles implicit demographic bias in large language models by introducing the DIF framework (Demographic Implicit Fairness), a scalable benchmark that couples sociodemographic personas with standard math and logic datasets and a null-model robustness test. It defines a pairwise, answer-level bias metric and uses a null-model comparison to distinguish genuine persona effects from prompt artifacts, with the key formulas $Bias = \frac{1}{N}\sum^{N}_{i=1}{\frac{|C_i \oplus C_b|}{|C_i \cap C_b|}}$ and $DIF = \max(0, 1-\text{Bias})$. Empirical results across open-weight LLMs (e.g., Llama-3, Mistral, Phi, Gemma) and datasets (GSM-MC, MathQA, DeepMath) reveal an inverse relationship between problem-solving accuracy and implicit bias, and demonstrate the robustness of DIF to cross-model comparisons while highlighting dataset- and prompt-dependent variations. The work provides a practical, interpretable tool for cross-model bias assessment and lays groundwork for mitigation strategies in deployments involving demographic-sensitive tasks, while acknowledging scope limitations and ethical considerations surrounding bias measurement.

Abstract

As Large Language Models (LLMs) have risen in prominence over the past few years, there has been concern over the potential biases in LLMs inherited from the training data. Previous studies have examined how LLMs exhibit implicit bias, such as when response generation changes when different social contexts are introduced. We argue that this implicit bias is not only an ethical, but also a technical issue, as it reveals an inability of LLMs to accommodate extraneous information. However, unlike other measures of LLM intelligence, there are no standard methods to benchmark this specific subset of LLM bias. To bridge this gap, we developed a method for calculating an easily interpretable benchmark, DIF (Demographic Implicit Fairness), by evaluating preexisting LLM logic and math problem datasets with sociodemographic personas, which is combined with a statistical robustness check using a null model. We demonstrate that this method can validate the presence of implicit bias in LLM behavior and find an novel inverse trend between question answering accuracy and implicit bias, supporting our argument.

DIF: A Framework for Benchmarking and Verifying Implicit Bias in LLMs

TL;DR

This paper tackles implicit demographic bias in large language models by introducing the DIF framework (Demographic Implicit Fairness), a scalable benchmark that couples sociodemographic personas with standard math and logic datasets and a null-model robustness test. It defines a pairwise, answer-level bias metric and uses a null-model comparison to distinguish genuine persona effects from prompt artifacts, with the key formulas and . Empirical results across open-weight LLMs (e.g., Llama-3, Mistral, Phi, Gemma) and datasets (GSM-MC, MathQA, DeepMath) reveal an inverse relationship between problem-solving accuracy and implicit bias, and demonstrate the robustness of DIF to cross-model comparisons while highlighting dataset- and prompt-dependent variations. The work provides a practical, interpretable tool for cross-model bias assessment and lays groundwork for mitigation strategies in deployments involving demographic-sensitive tasks, while acknowledging scope limitations and ethical considerations surrounding bias measurement.

Abstract

As Large Language Models (LLMs) have risen in prominence over the past few years, there has been concern over the potential biases in LLMs inherited from the training data. Previous studies have examined how LLMs exhibit implicit bias, such as when response generation changes when different social contexts are introduced. We argue that this implicit bias is not only an ethical, but also a technical issue, as it reveals an inability of LLMs to accommodate extraneous information. However, unlike other measures of LLM intelligence, there are no standard methods to benchmark this specific subset of LLM bias. To bridge this gap, we developed a method for calculating an easily interpretable benchmark, DIF (Demographic Implicit Fairness), by evaluating preexisting LLM logic and math problem datasets with sociodemographic personas, which is combined with a statistical robustness check using a null model. We demonstrate that this method can validate the presence of implicit bias in LLM behavior and find an novel inverse trend between question answering accuracy and implicit bias, supporting our argument.
Paper Structure (15 sections, 2 equations, 1 figure, 8 tables)

This paper contains 15 sections, 2 equations, 1 figure, 8 tables.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Bias Benchmarks
  4. Mathematical Reasoning as a Bias Probe
  5. Persona and Prompt-Induced Bias
  6. Datasets
  7. Methodology
  8. LLM Comparison and Analysis
  9. Bias of different models
  10. Validating the significance of implicit bias
  11. Temperature and bias
  12. Conclusion
  13. Future Work
  14. Appendix
  15. Temperature and bias

Figures (1)

  • Figure 1: LLM intelligence (measured as number of questions correctly answered using the baseline persona) versus raw bias scores, for every model on GSM-MC ($R^2=0.66$), MathQA ($R^2=0.04$), and DeepMath ($R^2=0.49$). There is a negative correlation between intelligence and bias present for each dataset.