Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Gender Bias of LLMs in Making Morality Judgements

Divij Bajaj, Yuanyuan Lei, Jonathan Tong, Ruihong Huang

TL;DR

This work investigates whether current closed and open-source LLMs possess gender bias, especially when asked to give moral opinions, and explores real-world situations where LLMs reveal biases in moral decision-making.

Abstract

Large Language Models (LLMs) have shown remarkable capabilities in a multitude of Natural Language Processing (NLP) tasks. However, these models are still not immune to limitations such as social biases, especially gender bias. This work investigates whether current closed and open-source LLMs possess gender bias, especially when asked to give moral opinions. To evaluate these models, we curate and introduce a new dataset GenMO (Gender-bias in Morality Opinions) comprising parallel short stories featuring male and female characters respectively. Specifically, we test models from the GPT family (GPT-3.5-turbo, GPT-3.5-turbo-instruct, GPT-4-turbo), Llama 3 and 3.1 families (8B/70B), Mistral-7B and Claude 3 families (Sonnet and Opus). Surprisingly, despite employing safety checks, all production-standard models we tested display significant gender bias with GPT-3.5-turbo giving biased opinions in 24% of the samples. Additionally, all models consistently favour female characters, with GPT showing bias in 68-85% of cases and Llama 3 in around 81-85% instances. Additionally, our study investigates the impact of model parameters on gender bias and explores real-world situations where LLMs reveal biases in moral decision-making.

Evaluating Gender Bias of LLMs in Making Morality Judgements

TL;DR

This work investigates whether current closed and open-source LLMs possess gender bias, especially when asked to give moral opinions, and explores real-world situations where LLMs reveal biases in moral decision-making.

Abstract

Large Language Models (LLMs) have shown remarkable capabilities in a multitude of Natural Language Processing (NLP) tasks. However, these models are still not immune to limitations such as social biases, especially gender bias. This work investigates whether current closed and open-source LLMs possess gender bias, especially when asked to give moral opinions. To evaluate these models, we curate and introduce a new dataset GenMO (Gender-bias in Morality Opinions) comprising parallel short stories featuring male and female characters respectively. Specifically, we test models from the GPT family (GPT-3.5-turbo, GPT-3.5-turbo-instruct, GPT-4-turbo), Llama 3 and 3.1 families (8B/70B), Mistral-7B and Claude 3 families (Sonnet and Opus). Surprisingly, despite employing safety checks, all production-standard models we tested display significant gender bias with GPT-3.5-turbo giving biased opinions in 24% of the samples. Additionally, all models consistently favour female characters, with GPT showing bias in 68-85% of cases and Llama 3 in around 81-85% instances. Additionally, our study investigates the impact of model parameters on gender bias and explores real-world situations where LLMs reveal biases in moral decision-making.

Paper Structure

This paper contains 28 sections, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dataset Curation
  4. Moral Stories
  5. ETHICS
  6. Social Chemistry
  7. Data Preparation
  8. Data Annotation
  9. Evaluating Gender Bias of LLMs
  10. Do LLMs contain gender bias in moral opinions?
  11. Are LLMs biased towards one gender?
  12. Where is the bias more prominent?
  13. Is gender bias dependent on model size?
  14. Ablation Studies
  15. Effect of insignificant variations in prompt
  16. ...and 13 more sections

Figures (3)

  • Figure 1: Examples of parallel stories from our dataset
  • Figure 2: Top: A female-inclined response by GPT-3.5-turbo. Bottom: An example of a male-favouring response generated by Llama3.1-70B. The number of male-inclined responses is significantly less than the female-favouring responses for all models we evaluated.
  • Figure 3: Examples of gender bias in Claude3-Opus.