The Need for a Socially-Grounded Persona Framework for User Simulation

TL;DR

The paper tackles the limitation of demographic-based synthetic personas in social simulations, proposing SCOPE, a sociopsychological framework with $141$ attributes across $8$ facets, split into conditioning and evaluation categories. By collecting a rich, human-grounded dataset from $124$ participants and evaluating seven model families, the authors show that demographic information explains only about $1.5\%$ of variance in human responses, and that incorporating sociopsychological structure improves behavioral alignment while reducing demographic bias. Non-demographic personas based on values and identity can match or exceed fully conditioned personas with substantially lower bias, and SCOPE can augment existing persona pipelines (e.g., Nemotron) to boost external benchmark performance such as SimBench. The results advocate a shift from demographic proxies to structured, human-grounded facets for more realistic, fair, and generalizable AI social simulations, with practical guidance on when to use richer conditioning versus lean, non-demographic prompts. Overall, SCOPE offers a concrete, actionable method for constructing and evaluating sociologically plausible personas in NLP systems.

Abstract

Synthetic personas are widely used to condition large language models (LLMs) for social simulation, yet most personas are still constructed from coarse sociodemographic attributes or summaries. We revisit persona creation by introducing SCOPE, a socially grounded framework for persona construction and evaluation, built from a 141-item, two-hour sociopsychological protocol collected from 124 U.S.-based participants. Across seven models, we find that demographic-only personas are a structural bottleneck: demographics explain only ~1.5% of variance in human response similarity. Adding sociopsychological facets improves behavioral prediction and reduces over-accentuation, and non-demographic personas based on values and identity achieve strong alignment with substantially lower bias. These trends generalize to SimBench (441 aligned questions), where SCOPE personas outperform default prompting and NVIDIA Nemotron personas, and SCOPE augmentation improves Nemotron-based personas. Our results indicate that persona quality depends on sociopsychological structure rather than demographic templates or summaries.

Figures (3)

• Figure 1: End-to-end pipeline of our persona construction and evaluation framework. The process consists of four major stages: (1) Rich Persona Collection: a two-hour, 141-item sociopsychological survey capturing eight behavioral facets across 124 participants; (2) Persona Construction Cases: structured persona representations generated by selectively combining demographic, behavioral, psychological, and narrative facets, as well as AI-generated summaries; (3) Synthetic Persona Evaluation: measuring accuracy, similarity to human responses, and demographic accentuation; and
• Figure 2: Line graph denotes the correlation score of all seven LLM modes.
• Figure 3: Figure illustrates the facet used for constructing the persona to the ones used for evaluation as a baseline in creating better personas.