Moral Susceptibility and Robustness under Persona Role-Play in Large Language Models

TL;DR

This work analyzes how persona role-play influences moral judgments in large language models by linking prompts to the Moral Foundations Questionnaire (MFQ). It defines two metrics, moral robustness ($R$) and moral susceptibility ($S$), derived from within- and across-persona MFQ variability, and analyzes these across model families and sizes. The findings show that model family explains most of the variance in robustness, with Claude models typically most robust and Grok least, while susceptibility increases with model size within families and correlates positively with robustness, especially at the family level. By presenting foundation- and persona-level MFQ profiles, the paper offers a principled framework for comparing moral behavior across models, aiding deployment and alignment research.

Abstract

Large language models (LLMs) increasingly operate in social contexts, motivating analysis of how they express and shift moral judgments. In this work, we investigate the moral response of LLMs to persona role-play, prompting a LLM to assume a specific character. Using the Moral Foundations Questionnaire (MFQ), we introduce a benchmark that quantifies two properties: moral susceptibility and moral robustness, defined from the variability of MFQ scores across and within personas, respectively. We find that, for moral robustness, model family accounts for most of the variance, while model size shows no systematic effect. The Claude family is, by a significant margin, the most robust, followed by Gemini and GPT-4 models, with other families exhibiting lower robustness. In contrast, moral susceptibility exhibits a mild family effect but a clear within-family size effect, with larger variants being more susceptible. Moreover, robustness and susceptibility are positively correlated, an association that is more pronounced at the family level. Additionally, we present moral foundation profiles for models without persona role-play and for personas averaged across models. Together, these analyses provide a systematic view of how persona conditioning shapes moral behavior in large language models.

Figures (6)

• Figure 1: Left: summary of our data collection pipeline: we elicit models to respond to the MFQ conditioned to a persona. Right: summary of our benchmark pipeline: robustness, Eq. \ref{['eq:robustness']}, and susceptibility, Eq. \ref{['eq:overall-susceptibility']}, are computed from across and within persona variability in MFQ scores.
• Figure 2: Moral foundation profile across models with no-persona role-play (self) together with the average over all models. Points show mean rating and error bars denote standard errors across questions within each foundation. See Table \ref{['tab:moral_foundations_profiles']} for exact values.
• Figure 3: Moral foundation profiles for fourteen randomly selected personas averaged across models together with the average over all personas and models. Points show mean rating and error bars denote standard errors across questions within each foundation. See Table \ref{['tab:persona_moral_foundations_profiles']} for exact values.
• Figure 4: Left: moral robustness, Eq. \ref{['eq:robustness']}: higher values indicate greater MFQ rating stability. Right: moral susceptibility, Eq. \ref{['eq:overall-susceptibility']}: higher values indicate larger persona-driven shifts in MFQ scores.
• Figure 5: Moral robustness foundation profile across models, Eq. \ref{['eq:robustness']}: higher values indicate greater MFQ rating stability.