The Personality Trap: How LLMs Embed Bias When Generating Human-Like Personas

TL;DR

This work interrogates the biases that arise when large language models generate synthetic populations conditioned solely on personality questionnaire responses. It employs a multi-model pipeline (GPT-3.5, GPT-4o, Claude-3.5-s, LLaMa-3B, LLaMa-70B) to map EPQR-A scores into demographic personas and to test the effects of extreme trait anchoring (MaxN, MaxP) on representativeness and trait fidelity. The study finds strong WEIRD biases across models and, in some cases, overrepresentation of LGBTQ+ identities when Psychoticism is maximized, highlighting substantial risks of stereotyping. Nevertheless, the results also show that LLM-generated personas reasonably preserve the input personality profiles, suggesting potential utility for psychometric simulations if bias and ethical considerations are carefully managed.

Abstract

This paper examines biases in large language models (LLMs) when generating synthetic populations from responses to personality questionnaires. Using five LLMs, we first assess the representativeness and potential biases in the sociodemographic attributes of the generated personas, as well as their alignment with the intended personality traits. While LLMs successfully reproduce known correlations between personality and sociodemographic variables, all models exhibit pronounced WEIRD (western, educated, industrialized, rich and democratic) biases, favoring young, educated, white, heterosexual, Western individuals with centrist or progressive political views and secular or Christian beliefs. In a second analysis, we manipulate input traits to maximize Neuroticism and Psychoticism scores. Notably, when Psychoticism is maximized, several models produce an overrepresentation of non-binary and LGBTQ+ identities, raising concerns about stereotyping and the potential pathologization of marginalized groups. Our findings highlight both the potential and the risks of using LLMs to generate psychologically grounded synthetic populations.

Figures (2)

• Figure 1: Overview of the experimental pipeline. Our experiment (right square) builds on the English version of the EPQR-A responses from 826 simulated personas reported in ferreira2025well (left square), which were used as generation input. Five LLMs were then prompted to generate 826 synthetic personas and provide sociodemographic attributes while reflecting the input personality traits. Subsequently, the LLMs were tasked to answer both the EPQR-A and BFI with the generated personalities. In a further step, the LLMs generated another 826 synthetic personas based on extreme trait manipulations, again producing sociodemographic attributes consistent with the input traits. Potential biases in the representativeness of the sample populations were assessed by analysing the distributions of these attributes. Divergence, accuracy, and error analysis were computed between the input responses and those newly generated. In addition, correlations between the EPQR-A and the BFI were examined, along with internal consistency measured using Cronbach’s alpha.
• Figure 2: Words cloud comparison between description from GPT-4o Base (blue) and from GPT-4o MaxP (red). The size of words is proportional to the absolute difference in frequency: words more frequent in GPT-4o Base compared to GPT-4o MaxP are colored in blue (red in the opposite case).