Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring the Personality Traits of LLMs through Latent Features Steering

Shu Yang, Shenzhe Zhu, Liang Liu, Lijie Hu, Mengdi Li, Di Wang

TL;DR

The paper investigates how LLMs acquire and express personality by leveraging a social-determinism framework that separates long-term background factors (encoded in model parameters) from short-term pressures (prompts and context). It introduces a training-free latent-feature steering approach: long-term factors are decoded with Sparse Autoencoders (SAEs) on activations, while short-term pressures are captured with representation-based directions, then steered through residual-stream modifications and final-token activations, formalized as $ \mathbf{R}^l_{:,:t-1,:} \leftarrow \mathbf{R}^l_{:,:t-1,:} + c f_b^m$ and $ h_l(t-1) \leftarrow h_l(t-1) + c f_p^n $. The framework is evaluated using TRAIT-based personality assessments and SafetyBench across Gemma models, revealing that background factors can alter safety and bias and that model size modulates trait stability and sensitivity to prompts. The results demonstrate a feasible, training-free path to controllable LLM personality with important safety trade-offs, offering implications for personalized yet responsibly aligned AI systems.

Abstract

Large language models (LLMs) have significantly advanced dialogue systems and role-playing agents through their ability to generate human-like text. While prior studies have shown that LLMs can exhibit distinct and consistent personalities, the mechanisms through which these models encode and express specific personality traits remain poorly understood. To address this, we investigate how various factors, such as cultural norms and environmental stressors, encoded within LLMs, shape their personality traits, guided by the theoretical framework of social determinism. Inspired by related work on LLM interpretability, we propose a training-free approach to modify the model's behavior by extracting and steering latent features corresponding to factors within the model, thereby eliminating the need for retraining. Furthermore, we analyze the implications of these factors for model safety, focusing on their impact through the lens of personality.

Exploring the Personality Traits of LLMs through Latent Features Steering

TL;DR

The paper investigates how LLMs acquire and express personality by leveraging a social-determinism framework that separates long-term background factors (encoded in model parameters) from short-term pressures (prompts and context). It introduces a training-free latent-feature steering approach: long-term factors are decoded with Sparse Autoencoders (SAEs) on activations, while short-term pressures are captured with representation-based directions, then steered through residual-stream modifications and final-token activations, formalized as and . The framework is evaluated using TRAIT-based personality assessments and SafetyBench across Gemma models, revealing that background factors can alter safety and bias and that model size modulates trait stability and sensitivity to prompts. The results demonstrate a feasible, training-free path to controllable LLM personality with important safety trade-offs, offering implications for personalized yet responsibly aligned AI systems.

Abstract

Large language models (LLMs) have significantly advanced dialogue systems and role-playing agents through their ability to generate human-like text. While prior studies have shown that LLMs can exhibit distinct and consistent personalities, the mechanisms through which these models encode and express specific personality traits remain poorly understood. To address this, we investigate how various factors, such as cultural norms and environmental stressors, encoded within LLMs, shape their personality traits, guided by the theoretical framework of social determinism. Inspired by related work on LLM interpretability, we propose a training-free approach to modify the model's behavior by extracting and steering latent features corresponding to factors within the model, thereby eliminating the need for retraining. Furthermore, we analyze the implications of these factors for model safety, focusing on their impact through the lens of personality.

Paper Structure

This paper contains 19 sections, 2 equations, 8 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Social Determinism in LLM Personality
  4. Tracing the Origins of Personality in LLMs through Interpretable Features
  5. Experiment setup
  6. Analyzing Steering Effects Through Social Bias
  7. Experimental Results
  8. Safety and Personality
  9. Conclusion
  10. Limitations
  11. Linear Representations in LLMs.
  12. Extract and Steering Latent Features with Sparse Autoencoders (SAEs)
  13. Details of personality traits and factors
  14. Big Five Inventory (BFI) and Short Dark Triad (SD-3)
  15. Short-term Pressure
  16. ...and 4 more sections

Figures (8)

  • Figure 1: The procedure of LLM personality exploring through features steering.
  • Figure 2: The logit difference between paired social attribute under steered and original activations. SS: Socioeconomic Status; SI: Social Ideology.
  • Figure 3:
  • Figure 4:
  • Figure 6:
  • ...and 3 more figures