DPRF: A Generalizable Dynamic Persona Refinement Framework for Optimizing Behavior Alignment Between Personalized LLM Role-Playing Agents and Humans

TL;DR

DPRF introduces a three-agent, iterative framework to align LLM RPAs with target individuals by detecting cognitive divergences between generated behavior and human ground truth and refining the persona accordingly. It contrasts free-form and theory-grounded ToM analyses to diagnose divergences and guide refinement, demonstrating generalizability across four tasks and five models. The results show improved semantic and lexical alignment over baseline personas, with task-dependent benefits and identifiable boundary conditions in highly dynamic scenarios. The approach offers a data-driven, model-agnostic path toward more faithful and adaptable personalized AI agents with wide-ranging applications in simulations, user studies, and personalized assistance.

Abstract

The emerging large language model role-playing agents (LLM RPAs) aim to simulate individual human behaviors, but the persona fidelity is often undermined by manually-created profiles (e.g., cherry-picked information and personality characteristics) without validating the alignment with the target individuals. To address this limitation, our work introduces the Dynamic Persona Refinement Framework (DPRF). DPRF aims to optimize the alignment of LLM RPAs' behaviors with those of target individuals by iteratively identifying the cognitive divergence, either through free-form or theory-grounded, structured analysis, between generated behaviors and human ground truth, and refining the persona profile to mitigate these divergences. We evaluate DPRF with five LLMs on four diverse behavior-prediction scenarios: formal debates, social media posts with mental health issues, public interviews, and movie reviews. DPRF can consistently improve behavioral alignment considerably over baseline personas and generalizes across models and scenarios. Our work provides a robust methodology for creating high-fidelity persona profiles and enhancing the validity of downstream applications, such as user simulation, social studies, and personalized AI.

Figures (6)

• Figure 1: The architecture of our DPRF framework, which constitutes an iterative process with three primary components: the role-playing agent, behavior analysis agent, and persona refinement agent.
• Figure 2: Sentence Embedding Similarity on small models across different datasets
• Figure 3: Bertscore-f1 on small models across different datasets
• Figure 4: RougeL-f1 on small models across different datasets
• Figure 5: Claude ablation result with structured analysis prompt