Linear Personality Probing and Steering in LLMs: A Big Five Study

TL;DR

This work investigates whether linear directions in LLM activation space aligned with the Big Five personality traits can be used to probe and steer model behavior. By generating 406 trait-annotated character profiles via IPIP, sampling activations across Llama 3.3 70B, and learning per-layer trait directions through regression, the authors demonstrate that trait-aligned directions meaningfully probe personality and generalize to trait-relevant adjectives. Steering experiments show reliable shifts in forced-choice tasks but reveal strong context-dependency, with explicit prompts often overriding vector-based interventions in open-ended generation. The findings highlight both the promise and limits of inference-time, linear personality control and suggest richer, multi-axis representations may be needed for robust real-world manipulation. Overall, the study links psychometric theory to LLM behavior, offering principled probing methods and clarifying the contextual constraints of steering in complex language tasks.

Abstract

Large language models (LLMs) exhibit distinct and consistent personalities that greatly impact trust and engagement. While this means that personality frameworks would be highly valuable tools to characterize and control LLMs' behavior, current approaches remain either costly (post-training) or brittle (prompt engineering). Probing and steering via linear directions has recently emerged as a cheap and efficient alternative. In this paper, we investigate whether linear directions aligned with the Big Five personality traits can be used for probing and steering model behavior. Using Llama 3.3 70B, we generate descriptions of 406 fictional characters and their Big Five trait scores. We then prompt the model with these descriptions and questions from the Alpaca questionnaire, allowing us to sample hidden activations that vary along personality traits in known, quantifiable ways. Using linear regression, we learn a set of per-layer directions in activation space, and test their effectiveness for probing and steering model behavior. Our results suggest that linear directions aligned with trait-scores are effective probes for personality detection, while their steering capabilities strongly depend on context, producing reliable effects in forced-choice tasks but limited influence in open-ended generation or when additional context is present in the prompt.

Figures (6)

• Figure 1: Overview diagram showing how we extract linear directions aligned with Big Five scores to probe and steer LLM personality traits.
• Figure 2: Big Five scores for selected characters from the dataset. Percentiles calculated from 300k human responses from openpsychometrics.com openpsychometrics-bigfive are overlaid for reference.
• Figure 3: Pairwise inner products between linear directions grouped by trait score in layer 18. Activations were extracted from three positions: the mean of the input prompt, the last token, and the mean of the generated response. The regression-based approach in panel a yields directions with low cross-talk and reasonable alignment for the same trait. Linear directions capturing the highest variance, derived from SVD in panels b and c are almost normal to the directions derived from regression at the last token. Highest variance directions in c derived from different token positions are almost normal as well, different from their regression counterparts.
• Figure 4: Projections of the hidden activation of the last token of the prompt in Listing \ref{['lst:prompt-adjectives']} onto the directions obtained from regression on the activations of the last token. ROC curves show for each layer how well the different adjectives are separated.
• Figure 5: Steering effects on forced-choice personality assessment across different extraction methods and prompt conditions. Panels show the fraction of extraverted (green) versus introverted (red) statements selected as a function of steering strength $\alpha$. (a) Steering with vectors derived from regression on mean input prompt activations produces monotonic transitions between personality extremes. (b) Steering with vectors derived from mean generated answer activations shows more gradual but less reliable transitions. (c--d) When character descriptions are added to the system prompt, steering effects disappear entirely for both extraction methods—the explicit personality context overrides the steering intervention, and responses remain consistent with the character description regardless of $\alpha$. Gray regions indicate invalid or off-list responses. Step-like transitions reflect the discrete nature of the forced-choice task, where continuous steering parameters map to categorical selection patterns through threshold effects.