Can LLMs Express Personality Across Cultures? Introducing CulturalPersonas for Evaluating Trait Alignment

TL;DR

CulturalPersonas introduces a culturally grounded benchmark to evaluate LLMs’ personality expression across six countries and five Big Five traits, using 3,000 scenario-based prompts and two evaluation formats (MCS and OEG). The approach grounds prompts in country-specific norms via a retrieval-augmented generation pipeline and validates cultural fidelity with human annotators, achieving closer alignment to human trait distributions than prior benchmarks as measured by $W$ and $KS$, while also increasing lexical expressivity in open-ended responses. The study demonstrates model- and trait-specific variability in cultural adaptation and highlights the positive impact of cultural priming on alignment. This work provides a path toward globally adaptive, socially intelligent LLMs, with implications for safety, inclusivity, and user-tailored interaction across diverse cultural contexts.

Abstract

As LLMs become central to interactive applications, ranging from tutoring to mental health, the ability to express personality in culturally appropriate ways is increasingly important. While recent works have explored personality evaluation of LLMs, they largely overlook the interplay between culture and personality. To address this, we introduce CulturalPersonas, the first large-scale benchmark with human validation for evaluating LLMs' personality expression in culturally grounded, behaviorally rich contexts. Our dataset spans 3,000 scenario-based questions across six diverse countries, designed to elicit personality through everyday scenarios rooted in local values. We evaluate three LLMs, using both multiple-choice and open-ended response formats. Our results show that CulturalPersonas improves alignment with country-specific human personality distributions (over a 20% reduction in Wasserstein distance across models and countries) and elicits more expressive, culturally coherent outputs compared to existing benchmarks. CulturalPersonas surfaces meaningful modulated trait outputs in response to culturally grounded prompts, offering new directions for aligning LLMs to global norms of behavior. By bridging personality expression and cultural nuance, we envision that CulturalPersonas will pave the way for more socially intelligent and globally adaptive LLMs.

Figures (11)

• Figure 1: Creation of CulturalPersonas involves: (1) retrieving 30 country-specific documents from Google Scholar and extracting cultural insights using GPT-4o and RAG; (2) contradiction detection and refinement via GPT-4o; (3) validation by two native annotators per country.
• Figure 2: Wasserstein distance and KS statistic metrics (average over OCEAN dimensions) for the MCS setting across all personality evaluators for GPT-4o-mini. Lower values indicate closer alignment with human distributions. Across all six benchmarked countries, CulturalPersonas yields the closest personality alignment.
• Figure 3: Pearson Correlation ($r$) of Human Annotated Likert Scores and Embedding-based Auto Mapping; values indicate high correlation between human and auto mapping methods.
• Figure 4: Wasserstein distances across OCEAN traits and models (avgerage over countries) for MCS. GPT-4o-mini performs best overall. CulturalPersonas reveals models struggle with traits: Agreeableness and Extraversion across cultures.
• Figure 5: Effect of cultural priming for Brazil across models and personality evaluators. Priming consistently reduces deviation across benchmarks. Bars show relative improvement in Wasserstein distances (%); * and ** denote $p < 0.05$ and $p < 0.01$ (Wilcoxon signed-rank test over 5 traits).