Hire Me or Not? Examining Language Model's Behavior with Occupation Attributes

TL;DR

This work tackles gender stereotypes in occupation-related decisions made by LLMs by introducing a multi-step verification framework grounded in the O*NET-SOC taxonomy. The method constructs a controlled dataset and uses three sequential tasks plus two metrics (Confirmation and Consistency) to audit bias without requiring model weights. Empirical results across RoBERTa-large, GPT-3.5-turbo, Llama2-70b-chat, and GPT-4o-mini reveal persistent gender biases with model-dependent patterns, suggesting current alignment methods may not fully mitigate bias and can induce new effects. The framework provides a practical bias auditing tool for production pipelines and points to needs in debiasing research and safer deployment practices.

Abstract

With the impressive performance in various downstream tasks, large language models (LLMs) have been widely integrated into production pipelines, like recruitment and recommendation systems. A known issue of models trained on natural language data is the presence of human biases, which can impact the fairness of the system. This paper investigates LLMs' behavior with respect to gender stereotypes, in the context of occupation decision making. Our framework is designed to investigate and quantify the presence of gender stereotypes in LLMs' behavior via multi-round question answering. Inspired by prior works, we construct a dataset by leveraging a standard occupation classification knowledge base released by authoritative agencies. We tested three LLMs (RoBERTa-large, GPT-3.5-turbo, and Llama2-70b-chat) and found that all models exhibit gender stereotypes analogous to human biases, but with different preferences. The distinct preferences of GPT-3.5-turbo and Llama2-70b-chat may imply the current alignment methods are insufficient for debiasing and could introduce new biases contradicting the traditional gender stereotypes.

Figures (6)

• Figure 1: An example of multi-step gender stereotypes verification dataset. The yellow outputs indicate that the model's behaviors have low Confirmation and high Consistency. The blue outputs indicate that the behaviors have high Confirmation and low Consistency.
• Figure 2: Multi-rounds of questions
• Figure 3: Confirmation metric (comparison between $Q_1$ and $Q_2$) for the three language models; lower values indicate gender bias.
• Figure 4: Consistency metric (comparison between $Q_2$ and $Q_3$) for the three language models; higher values indicate gender bias.
• Figure 5: Scatterplot of Confirmation ($Q_1Q_2$) vs. Consistency ($Q_2Q_3$) across three language models under different-gender settings.