Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Confidence Trap: Gender Bias and Predictive Certainty in LLMs

Ahmed Sabir, Markus Kängsepp, Rajesh Sharma

TL;DR

This work investigates how LLMs' predicted confidence aligns with human gender-bias judgments in pronoun-resolution tasks. It introduces Gender-ECE, a gender-aware calibration metric, and evaluates six open-weight LLMs on bias benchmarks including WinoBias, Winogender, GenderLex, and WinoQueer, revealing substantial miscalibration and gender gaps (e.g., Gemma-2-9B often worst; GPT-J-6B often best). The study shows calibration alone does not solve fairness issues, but post-hoc Beta calibration can improve reliability, and explicit contextual prompts reduce uncertainty. Overall, Gender-ECE provides a targeted tool for fairness-aware calibration assessment, guiding ethical deployment and highlighting directions for future bias-mitigation research.

Abstract

The increased use of Large Language Models (LLMs) in sensitive domains leads to growing interest in how their confidence scores correspond to fairness and bias. This study examines the alignment between LLM-predicted confidence and human-annotated bias judgments. Focusing on gender bias, the research investigates probability confidence calibration in contexts involving gendered pronoun resolution. The goal is to evaluate if calibration metrics based on predicted confidence scores effectively capture fairness-related disparities in LLMs. The results show that, among the six state-of-the-art models, Gemma-2 demonstrates the worst calibration according to the gender bias benchmark. The primary contribution of this work is a fairness-aware evaluation of LLMs' confidence calibration, offering guidance for ethical deployment. In addition, we introduce a new calibration metric, Gender-ECE, designed to measure gender disparities in resolution tasks.

The Confidence Trap: Gender Bias and Predictive Certainty in LLMs

TL;DR

This work investigates how LLMs' predicted confidence aligns with human gender-bias judgments in pronoun-resolution tasks. It introduces Gender-ECE, a gender-aware calibration metric, and evaluates six open-weight LLMs on bias benchmarks including WinoBias, Winogender, GenderLex, and WinoQueer, revealing substantial miscalibration and gender gaps (e.g., Gemma-2-9B often worst; GPT-J-6B often best). The study shows calibration alone does not solve fairness issues, but post-hoc Beta calibration can improve reliability, and explicit contextual prompts reduce uncertainty. Overall, Gender-ECE provides a targeted tool for fairness-aware calibration assessment, guiding ethical deployment and highlighting directions for future bias-mitigation research.

Abstract

The increased use of Large Language Models (LLMs) in sensitive domains leads to growing interest in how their confidence scores correspond to fairness and bias. This study examines the alignment between LLM-predicted confidence and human-annotated bias judgments. Focusing on gender bias, the research investigates probability confidence calibration in contexts involving gendered pronoun resolution. The goal is to evaluate if calibration metrics based on predicted confidence scores effectively capture fairness-related disparities in LLMs. The results show that, among the six state-of-the-art models, Gemma-2 demonstrates the worst calibration according to the gender bias benchmark. The primary contribution of this work is a fairness-aware evaluation of LLMs' confidence calibration, offering guidance for ethical deployment. In addition, we introduce a new calibration metric, Gender-ECE, designed to measure gender disparities in resolution tasks.
Paper Structure (16 sections, 9 equations, 4 figures, 11 tables)

This paper contains 16 sections, 9 equations, 4 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Datasets
  4. Methods
  5. Experimental Result
  6. Discussion
  7. Calibration
  8. Ablation Study
  9. Conclusion
  10. Ethical Statement
  11. Appendix
  12. Implantation details
  13. Limitations and Future Work
  14. Ablation Study
  15. Additional Information
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Reliability diagrams for the GenderLex dataset with the gender appearing at the end of the sentence having all context last cloze structure. Gemma-2-9B exhibits the highest ECE while GPT-J-6B is the most calibrated model.
  • Figure 2: (Top) Reliability diagrams for WinoBias. (Bottom) Reliability diagrams for Winogender. GPT-J-6B shows the most calibrated model performance, while Gemma-2-9B demonstrates the worst human-alignment and calibration.
  • Figure 3: Comparison of uncalibrated (Top) and Beta calibrated (Bottom) gender bias probabilities for WinoBias dataset. Instances are divided into 10 equal-width bins. The red bar shows the distance to perfectly calibrated probabilities in the bin, and the blue bar shows the accuracy. The histogram below the reliability diagram shows how many instances are in each bin.
  • Figure 4: Comparison of uncalibrated (Top) and Beta calibrated kull2017beta (Bottom) gender bias probabilities for WinoBias dataset. Instances are divided into 10 equal-size bins instead of equal-width bins. The red bar shows the distance to perfectly calibrated probabilities.