Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sociodemographic Bias in Language Models: A Survey and Forward Path

Vipul Gupta, Pranav Narayanan Venkit, Shomir Wilson, Rebecca J. Passonneau

TL;DR

This survey comprehensively maps a decade of sociodemographic bias research in NLP by framing the field with a three-pronged typology: (i) types of bias, (ii) methods to quantify bias, and (iii) debiasing strategies applied at finetuning, training, and inference. It evaluates intrinsic and extrinsic measurement approaches, highlights foundational datasets such as WEAT, CrowS-Pairs, and HolisticBias, and discusses the limitations and reliability of current metrics. The authors propose a 13-question checklist to guide robust, interdisciplinary, and reproducible bias research, and they identify training-time mitigation and expert-model techniques as promising directions. The work emphasizes practical impact, language diversity, and sociotechnical considerations, calling for broader collaboration beyond NLP to curb harms and improve equitable deployment of language technologies.

Abstract

Sociodemographic bias in language models (LMs) has the potential for harm when deployed in real-world settings. This paper presents a comprehensive survey of the past decade of research on sociodemographic bias in LMs, organized into a typology that facilitates examining the different aims: types of bias, quantifying bias, and debiasing techniques. We track the evolution of the latter two questions, then identify current trends and their limitations, as well as emerging techniques. To guide future research towards more effective and reliable solutions, and to help authors situate their work within this broad landscape, we conclude with a checklist of open questions.

Sociodemographic Bias in Language Models: A Survey and Forward Path

TL;DR

This survey comprehensively maps a decade of sociodemographic bias research in NLP by framing the field with a three-pronged typology: (i) types of bias, (ii) methods to quantify bias, and (iii) debiasing strategies applied at finetuning, training, and inference. It evaluates intrinsic and extrinsic measurement approaches, highlights foundational datasets such as WEAT, CrowS-Pairs, and HolisticBias, and discusses the limitations and reliability of current metrics. The authors propose a 13-question checklist to guide robust, interdisciplinary, and reproducible bias research, and they identify training-time mitigation and expert-model techniques as promising directions. The work emphasizes practical impact, language diversity, and sociotechnical considerations, calling for broader collaboration beyond NLP to curb harms and improve equitable deployment of language technologies.

Abstract

Sociodemographic bias in language models (LMs) has the potential for harm when deployed in real-world settings. This paper presents a comprehensive survey of the past decade of research on sociodemographic bias in LMs, organized into a typology that facilitates examining the different aims: types of bias, quantifying bias, and debiasing techniques. We track the evolution of the latter two questions, then identify current trends and their limitations, as well as emerging techniques. To guide future research towards more effective and reliable solutions, and to help authors situate their work within this broad landscape, we conclude with a checklist of open questions.
Paper Structure (44 sections, 3 figures, 2 tables)

This paper contains 44 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Understanding Bias
  3. Categories of Work on Bias in LMs
  4. Types of Bias - T1
  5. Quantifying Bias
  6. Distance-based metrics - Q1
  7. Distance in vector space.
  8. Probing metrics - Q2
  9. Performance-based metrics - Q3
  10. Prompt-based metrics - Q4
  11. Template-based methods.
  12. Counterfactual-based methods.
  13. Masking Sentences.
  14. Debiasing
  15. Debiasing during Finetuning - D1
  16. ...and 29 more sections

Figures (3)

  • Figure 1: This graph shows number of papers/articles published each year (from 2013 to 2023) in SCOPUS that contain the term ‘bias’ and ('nlp' or 'language models') in the title, abstract, or keywords.
  • Figure 2: Three broad categories of bias research, and the upper hierarchy of each category (T, Q, D).
  • Figure 3: Evolution of changes in methods to quantify LM bias and debiasing LMs over the past decade.