Improving Language Model Personas via Rationalization with Psychological Scaffolds

TL;DR

PB&J (Psychology of Behavior and Judgments) introduces a framework to improve language-model personas by generating post-hoc rationales for user judgments grounded in psychological scaffolds such as the Big 5, Schwartz values, and Primal World Beliefs. Rationales are produced as r_basic = LM_R(j, Q) and scaffolded variants r_ψ = LM_R(j, Q, ψ), then integrated into the persona prompt to guide responses. Empirical results on OpinionQA and MovieLens show that PB&J with scaffolded rationales consistently surpass baselines that use only demographics or judgments, and can match or approach performance when compared to human-written rationales. The work highlights that structured, theory-guided synthetic rationales can enhance personalization, even with limited judgments, and suggests practical potential for zero-shot personalization with thoughtful scaffolds, while acknowledging limitations in fidelity to actual user reasoning and biases in training data.

Abstract

Language models prompted with a user description or persona are being used to predict the user's preferences and opinions. However, existing approaches to building personas mostly rely on a user's demographic attributes and/or prior judgments, but not on any underlying reasoning behind a user's judgments. We introduce PB&J (Psychology of Behavior and Judgments), a framework that improves LM personas by incorporating potential rationales for why the user could have made a certain judgment. Our rationales are generated by a language model to explicitly reason about a user's behavior on the basis of their experiences, personality traits, or beliefs. Our method employs psychological scaffolds: structured frameworks such as the Big 5 Personality Traits or Primal World Beliefs to help ground the generated rationales in existing theories. Experiments on public opinion and movie preference prediction tasks demonstrate that language model personas augmented with PB&J rationales consistently outperform personas conditioned only on user demographics and / or judgments, including those that use a model's default chain-of-thought, which is not grounded in psychological theories. Additionally, our PB&J personas perform competitively with those using human-written rationales, suggesting the potential of synthetic rationales guided by existing theories.

Figures (5)

• Figure 1: Overview of the PB&J framework: A base persona comprising user Demographics and Judgments is augmented with post-hoc rationales of various Scaffolds generated by an LM. The updated persona is integrated into the system prompt of an LM to align predictions with user behavior.
• Figure 2: Example Task: Shown here are example inputs and outputs for OpinionQA and MovieLens datasets respectively. Each of these instances have corresponding user selected answers.
• Figure 3: Performance as a function of the number of user judgments:PB&J outperforms baselines across all settings, providing substantial gains even with minimal judgments. All results use GPT-4.
• Figure 4: PB&J's improvements over Demographics + Judgments across education, race, income, and gender: Subgroups marked with * indicate significant improvements (p < 0.05). All results use GPT-4.
• Figure 5: PB&J's improvements over Demographics + Judgments across all demographics: Subgroups marked with * indicate significant improvements (p < 0.05). All results use GPT-4.