PTCBENCH: Benchmarking Contextual Stability of Personality Traits in LLM Systems

TL;DR

PTCBench addresses the problem of context-driven personality dynamics in LLMs by introducing a psychologically grounded benchmark that quantifies trait changes across 12 external contexts using the NEO-FFI and the Big Five dimensions $O$, $C$, $E$, $A$, and $N$. The approach systematically evaluates 4 foundation models and 2 agent frameworks, yielding 39,240 trait records across 244 preset configurations, and reveals that location and life-event contexts can induce substantial trait shifts, with Divorce and Unemployment producing the largest deviations and measurable changes in reasoning. The study demonstrates architecture-dependent stability, with foundation models showing bounded adaptation and agentic systems like AutoGen exhibiting pronounced instability, and demonstrates that preset personality priors modulate these effects. Overall, PTCBench provides a scalable protocol for evaluating and guiding psychologically coherent LLM behavior in evolving environments, informing design choices for robust, trustworthy, and adaptable AI companions.

Abstract

With the increasing deployment of large language models (LLMs) in affective agents and AI systems, maintaining a consistent and authentic LLM personality becomes critical for user trust and engagement. However, existing work overlooks a fundamental psychological consensus that personality traits are dynamic and context-dependent. To bridge this gap, we introduce PTCBENCH, a systematic benchmark designed to quantify the consistency of LLM personalities under controlled situational contexts. PTCBENCH subjects models to 12 distinct external conditions spanning diverse location contexts and life events, and rigorously assesses the personality using the NEO Five-Factor Inventory. Our study on 39,240 personality trait records reveals that certain external scenarios (e.g., "Unemployment") can trigger significant personality changes of LLMs, and even alter their reasoning capabilities. Overall, PTCBENCH establishes an extensible framework for evaluating personality consistency in realistic, evolving environments, offering actionable insights for developing robust and psychologically aligned AI systems.

Figures (9)

• Figure 1: The typical elements to describe the comprehensive context for an incident.
• Figure 2: The overview of PTCBench.
• Figure 3: The personality trait of different LLMs and agents.
• Figure 4: Comparison of average personality trait changes in GPT-4o-mini.
• Figure 5: Comparison of average personality trait changes in Gemini 2.0 Flash.