Widespread Gender and Pronoun Bias in Moral Judgments Across LLMs

Abstract

Large language models (LLMs) are increasingly used to assess moral or ethical statements, yet their judgments may reflect social and linguistic biases. This work presents a controlled, sentence-level study of how grammatical person, number, and gender markers influence LLM moral classifications of fairness. Starting from 550 balanced base sentences from the ETHICS dataset, we generated 26 counterfactual variants per item, systematically varying pronouns and demographic markers to yield 14,850 semantically equivalent sentences. We evaluated six model families (Grok, GPT, LLaMA, Gemma, DeepSeek, and Mistral), and measured fairness judgments and inter-group disparities using Statistical Parity Difference (SPD). Results show statistically significant biases: sentences written in the singular form and third person are more often judged as "fair'', while those in the second person are penalized. Gender markers produce the strongest effects, with non-binary subjects consistently favored and male subjects disfavored. We conjecture that these patterns reflect distributional and alignment biases learned during training, emphasizing the need for targeted fairness interventions in moral LLM applications.

Figures (15)

• Figure 1: Accuracy of nine models across 27 subject variants. Rows are ordered by the top model’s accuracy (highest to lowest). The final row reports the per-model average accuracy.
• Figure 2: Rank-frequency heatmap across variants.
• Figure 3: Boxplots of error rates (%) across variants of the same base sentence.
• Figure 4: $\mathrm{SPD}(V_i \rightarrow V_j)$ between source variant $V_i$ and target variant $V_j$ computed with Grok 4 Fast Reasoning. Values > $0$ favor the row variant.
• Figure 5: Spearman correlations among models’ $\mathrm{SPD}$ variant rankings.