Evaluating Gender Bias in Large Language Models

TL;DR

Overall, the prompting method had a greater impact on gender distribution than the model selection itself, highlighting the complexity of addressing gender bias in LLMs and the importance of prompting in gender mapping.

Abstract

Gender bias in artificial intelligence has become an important issue, particularly in the context of language models used in communication-oriented applications. This study examines the extent to which Large Language Models (LLMs) exhibit gender bias in pronoun selection in occupational contexts. The analysis evaluates the models GPT-4, GPT-4o, PaLM 2 Text Bison and Gemini 1.0 Pro using a self-generated dataset. The jobs considered include a range of occupations, from those with a significant male presence to those with a notable female concentration, as well as jobs with a relatively equal gender distribution. Three different sentence processing methods were used to assess potential gender bias: masked tokens, unmasked sentences, and sentence completion. In addition, the LLMs suggested names of individuals in specific occupations, which were then examined for gender distribution. The results show a positive correlation between the models' pronoun choices and the gender distribution present in U.S. labor force data. Female pronouns were more often associated with female-dominated occupations, while male pronouns were more often associated with male-dominated occupations. Sentence completion showed the strongest correlation with actual gender distribution, while name generation resulted in a more balanced 'politically correct' gender distribution, albeit with notable variations in predominantly male or female occupations. Overall, the prompting method had a greater impact on gender distribution than the model selection itself, highlighting the complexity of addressing gender bias in LLMs. The findings highlight the importance of prompting in gender mapping.

Figures (12)

• Figure 1: Example sentences from the dataset
• Figure 2: Gender proportion of selected professions bls2024 is aggregated into equal-sized bins of 10% increments, including the upper bound. Gender is depicted on a scale from 0 to 1, where 0 represents 100% male and 1 represents 100% female, with 0.5 indicating an equal male-female distribution.
• Figure 3: Flowcharts of the evaluation process: (a) for the three sentence processing pipelines (masked tokens, unmasked sentences, and sentence completion) and (b) for the name generation method.
• Figure 4: Example of masked tokens
• Figure 5: Example of unmasked sentences