Persona Generators: Generating Diverse Synthetic Personas at Scale

TL;DR

The paper tackles stress-testing AI-human interaction by shifting from fidelity to representative coverage of diverse user populations. It introduces Persona Generators, a two-stage, code-optimized mechanism that generates populations of synthetic personas tailored to arbitrary contexts and evaluated with Concordia simulations. An AlphaEvolve-driven loop mutates the generator code to maximize six diversity metrics, achieving substantial improvements over baselines in coverage and long-tail representation, with demonstrated generalization to held-out contexts and downstream tasks. The work offers a scalable, lightweight approach for on-demand diverse population synthesis, with potential applications in safety testing, red-teaming, and robust system design, while noting open challenges in measuring open-ended behavioral diversity and mutation-prompts biases.

Abstract

Evaluating AI systems that interact with humans requires understanding their behavior across diverse user populations, but collecting representative human data is often expensive or infeasible, particularly for novel technologies or hypothetical future scenarios. Recent work in Generative Agent-Based Modeling has shown that large language models can simulate human-like synthetic personas with high fidelity, accurately reproducing the beliefs and behaviors of specific individuals. However, most approaches require detailed data about target populations and often prioritize density matching (replicating what is most probable) rather than support coverage (spanning what is possible), leaving long-tail behaviors underexplored. We introduce Persona Generators, functions that can produce diverse synthetic populations tailored to arbitrary contexts. We apply an iterative improvement loop based on AlphaEvolve, using large language models as mutation operators to refine our Persona Generator code over hundreds of iterations. The optimization process produces lightweight Persona Generators that can automatically expand small descriptions into populations of diverse synthetic personas that maximize coverage of opinions and preferences along relevant diversity axes. We demonstrate that evolved generators substantially outperform existing baselines across six diversity metrics on held-out contexts, producing populations that span rare trait combinations difficult to achieve in standard LLM outputs.

Figures (23)

• Figure 1: Overview of the method. First, we generate questionnaires containing specific contexts $c$, dimensions $\mathcal{D}$ (diversity axes) and questions $\mathcal{I}$ (items). The Persona Generator $G_{\phi, \theta}(c, \mathcal{D}, N)$ uses the context and dimensions as inputs to create a population of synthetic personas $\mathcal{P}$. These personas are evaluated in Concordia simulations $Z = \Psi(\mathcal{P,I})$ where they answer the questionnaire items. We measure the diversity of their responses $\mathcal{M(Z)}$ and use AlphaEvolve to iteratively optimize the Persona Generator code $\phi$.
• Figure 2: Two stage Persona Generator. The Persona Generator $G_{\phi, \theta}(c, \mathcal{D}, N)$ works in two stages: the autoregressive stage 1 generates high-level descriptors for each persona, then the parallel stage 2 expands the high level descriptions of each persona by generating additional details.
• Figure 3: Evolution of Persona Generator performance. The top panel reports the mean score across six evaluation metrics, while the bottom panel reports coverage, showing that the optimized generator reaches over $80\%$ coverage on the test set. Each point corresponds to an evolved generator evaluated on 40 questionnaires (training plus validation sets, 25 personas each). The solid curve tracks the best-performing generator over time, and the dotted curves show its generalization on 10 held-out test questionnaires.
• Figure 4: Diversity for Downstream Tasks. The top figure shows diversity on comedy writing downstream tasks of the best evolved solutions by AlphaEvolve, and the bottom figure shows diversity for conflict resolution. While results are somewhat noisier than diversity measured with questionnaire preferences, we see that evolved Persona Generators perform better than the baselines.
• Figure 5: Questionnaire Generator. The questionnaire generator takes a short description of the target context and some few-shot example questionnaires. It then expands the context $c$ and proposes diversity axes $\mathcal{D}$, and finally produces the questions (items) $\mathcal{I}$