Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes

Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Tong Yu, Hanieh Deilamsalehy, Ruiyi Zhang, Sungchul Kim, Franck Dernoncourt

TL;DR

The paper tackles the problem of stereotyping in large language models by introducing zero-shot self-debiasing, a prompting-based bias mitigation that requires no training data or parameter updates. It implements two strategies—explanation prompting and reprompting—evaluated on the BBQ bias benchmark across nine social groups, using a baseline single-letter answer format. Both methods reduce stereotyping, with reprompting achieving the strongest decrease in aggregate bias while maintaining or improving answer correctness; results are robust via bootstrap confidence intervals. This work demonstrates a modular, scalable approach to bias reduction in black-box LLMs and invites broader exploration of zero-shot debiasing across tasks and models.

Abstract

Large language models (LLMs) have shown remarkable advances in language generation and understanding but are also prone to exhibiting harmful social biases. While recognition of these behaviors has generated an abundance of bias mitigation techniques, most require modifications to the training data, model parameters, or decoding strategy, which may be infeasible without access to a trainable model. In this work, we leverage the zero-shot capabilities of LLMs to reduce stereotyping in a technique we introduce as zero-shot self-debiasing. With two approaches, self-debiasing via explanation and self-debiasing via reprompting, we show that self-debiasing can significantly reduce the degree of stereotyping across nine different social groups while relying only on the LLM itself and a simple prompt, with explanations correctly identifying invalid assumptions and reprompting delivering the greatest reductions in bias. We hope this work opens inquiry into other zero-shot techniques for bias mitigation.

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes

TL;DR

The paper tackles the problem of stereotyping in large language models by introducing zero-shot self-debiasing, a prompting-based bias mitigation that requires no training data or parameter updates. It implements two strategies—explanation prompting and reprompting—evaluated on the BBQ bias benchmark across nine social groups, using a baseline single-letter answer format. Both methods reduce stereotyping, with reprompting achieving the strongest decrease in aggregate bias while maintaining or improving answer correctness; results are robust via bootstrap confidence intervals. This work demonstrates a modular, scalable approach to bias reduction in black-box LLMs and invites broader exploration of zero-shot debiasing across tasks and models.

Abstract

Large language models (LLMs) have shown remarkable advances in language generation and understanding but are also prone to exhibiting harmful social biases. While recognition of these behaviors has generated an abundance of bias mitigation techniques, most require modifications to the training data, model parameters, or decoding strategy, which may be infeasible without access to a trainable model. In this work, we leverage the zero-shot capabilities of LLMs to reduce stereotyping in a technique we introduce as zero-shot self-debiasing. With two approaches, self-debiasing via explanation and self-debiasing via reprompting, we show that self-debiasing can significantly reduce the degree of stereotyping across nine different social groups while relying only on the LLM itself and a simple prompt, with explanations correctly identifying invalid assumptions and reprompting delivering the greatest reductions in bias. We hope this work opens inquiry into other zero-shot techniques for bias mitigation.
Paper Structure (19 sections, 1 equation, 3 figures, 6 tables)

This paper contains 19 sections, 1 equation, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Approach
  4. Bias Benchmark for QA (BBQ)
  5. Baseline Approach
  6. Self-Debiasing via Explanation
  7. Self-Debiasing via Reprompting
  8. Results
  9. Experimental Setup
  10. Baseline
  11. Bias Mitigation via Explanation
  12. Bias Mitigation via Reprompting
  13. Conclusion
  14. Limitations
  15. Ethical Considerations
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Distribution of bootstrapped bias scores for the baseline, self-debiasing via explanation, and self-debiasing via reprompting approaches. The dashed line shows the bias score without bootstrapping.
  • Figure 2: Effect of the temperature parameter on the distribution of bootstrapped bias scores for the baseline, self-debiasing via explanation, and self-debiasing via reprompting approaches. The bias scores are calculated over 250 randomly selected gender identity questions.
  • Figure 3: Distribution of bootstrapped bias scores for the baseline, self-debiasing via explanation, and self-debiasing via reprompting approaches. The dashed lines show the overall aggregate bias scores for each technique.