The Persona Paradox: Medical Personas as Behavioral Priors in Clinical Language Models

TL;DR

This work shows that personas function as behavioral priors that introduce context-dependent trade-offs rather than guarantees of safety or expertise, and finds systematic, context-dependent, and non-monotonic effects.

Abstract

Persona conditioning can be viewed as a behavioral prior for large language models (LLMs) and is often assumed to confer expertise and improve safety in a monotonic manner. However, its effects on high-stakes clinical decision-making remain poorly characterized. We systematically evaluate persona-based control in clinical LLMs, examining how professional roles (e.g., Emergency Department physician, nurse) and interaction styles (bold vs.\ cautious) influence behavior across models and medical tasks. We assess performance on clinical triage and patient-safety tasks using multidimensional evaluations that capture task accuracy, calibration, and safety-relevant risk behavior. We find systematic, context-dependent, and non-monotonic effects: Medical personas improve performance in critical care tasks, yielding gains of up to $\sim+20\%$ in accuracy and calibration, but degrade performance in primary-care settings by comparable margins. Interaction style modulates risk propensity and sensitivity, but it's highly model-dependent. While aggregated LLM-judge rankings favor medical over non-medical personas in safety-critical cases, we found that human clinicians show moderate agreement on safety compliance (average Cohen's $κ= 0.43$) but indicate a low confidence in 95.9\% of their responses on reasoning quality. Our work shows that personas function as behavioral priors that introduce context-dependent trade-offs rather than guarantees of safety or expertise. The code is available at https://github.com/rsinghlab/Persona\_Paradox.

Figures (7)

• Figure 1: Experimental framework for analyzing personas as behavioral priors in clinical LLMs. Personas are injected via system prompts (A). Models are evaluated on two clinical tasks, yielding decision labels, free-text justifications, and latent logit scores (B). Behavioral effects are quantified using automated metrics and assessed qualitatively through blinded LLM-based rankings, with validation by expert clinicians (C).
• Figure 2: Persona effects on Clinical Triage. Bars show $\Delta$ relative to no‑persona baseline. On average, medical Personas improve emergency performance but degrade primary care performance, with model‑dependent effects on consistency. Arrows represent the directionality of the metric. '*' represents statistical significance.
• Figure 3: Interaction style effects Risk Propensity (left) and Risk Sensitivity (right) on Clinical Triage.
• Figure 4: Performance on LLM-based evaluation. (a) LLM judges prefer medical personas across safety dimensions. (b) LLM Judges mirror context-dependent effects observed in justification quality rankings. (c) LLM judges perceive Cautious variants as safer than Bold. '*' represents statistical significance.
• Figure 5: Clinician preference statistics. (a) Task-specific confidence distribution. (b) Inter-annotator agreements.