Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rediscovering the Latent Dimensions of Personality with Large Language Models as Trait Descriptors

Joseph Suh, Suhong Moon, Minwoo Kang, David M. Chan

TL;DR

A novel approach is introduced that uncovers latent personality dimensions in LLMs by applying singular value de-composition (SVD) to the log-probabilities of trait-descriptive adjectives.

Abstract

Assessing personality traits using large language models (LLMs) has emerged as an interesting and challenging area of research. While previous methods employ explicit questionnaires, often derived from the Big Five model of personality, we hypothesize that LLMs implicitly encode notions of personality when modeling next-token responses. To demonstrate this, we introduce a novel approach that uncovers latent personality dimensions in LLMs by applying singular value de-composition (SVD) to the log-probabilities of trait-descriptive adjectives. Our experiments show that LLMs "rediscover" core personality traits such as extraversion, agreeableness, conscientiousness, neuroticism, and openness without relying on direct questionnaire inputs, with the top-5 factors corresponding to Big Five traits explaining 74.3% of the variance in the latent space. Moreover, we can use the derived principal components to assess personality along the Big Five dimensions, and achieve improvements in average personality prediction accuracy of up to 5% over fine-tuned models, and up to 21% over direct LLM-based scoring techniques.

Rediscovering the Latent Dimensions of Personality with Large Language Models as Trait Descriptors

TL;DR

A novel approach is introduced that uncovers latent personality dimensions in LLMs by applying singular value de-composition (SVD) to the log-probabilities of trait-descriptive adjectives.

Abstract

Assessing personality traits using large language models (LLMs) has emerged as an interesting and challenging area of research. While previous methods employ explicit questionnaires, often derived from the Big Five model of personality, we hypothesize that LLMs implicitly encode notions of personality when modeling next-token responses. To demonstrate this, we introduce a novel approach that uncovers latent personality dimensions in LLMs by applying singular value de-composition (SVD) to the log-probabilities of trait-descriptive adjectives. Our experiments show that LLMs "rediscover" core personality traits such as extraversion, agreeableness, conscientiousness, neuroticism, and openness without relying on direct questionnaire inputs, with the top-5 factors corresponding to Big Five traits explaining 74.3% of the variance in the latent space. Moreover, we can use the derived principal components to assess personality along the Big Five dimensions, and achieve improvements in average personality prediction accuracy of up to 5% over fine-tuned models, and up to 21% over direct LLM-based scoring techniques.
Paper Structure (22 sections, 2 equations, 5 figures, 5 tables)

This paper contains 22 sections, 2 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Using Language Models as Personality Descriptors
  3. Re-Discovering the Big Five Structure
  4. Personality Trait Prediction
  5. Experiments
  6. The Big Five Structure
  7. Personality Trait Prediction
  8. Further Directions
  9. Conclusion
  10. Log-probability Measurement Details
  11. Why are TDA Log-probabilities Meaningful?
  12. Trait Descriptive Adjectives
  13. Tokenization of Adjectives
  14. Decoding Parameter: Temperature
  15. Dataset
  16. ...and 7 more sections

Figures (5)

  • Figure 1: Overview of our approach. Given a set of personal stories and a list of TDAs, a language model computes the likelihood of each trait term describing the author of the story, through which we construct an observation matrix of such log-probabilities. Singular Value Decomposition (SVD) is then applied to this matrix, which yields (1) the loading matrix $V$ that captures the latent dimension structure and (2) the factor matrix $U$ that explains each story author's personality spectrum in the projected low-dimensional space. Results are compared against findings from psychometric literature and binary Big Five labels of authors, respectively.
  • Figure 2: A question-answer format prompt that asks for a single adjective describing the personality of the author of the given personal story. A quotation mark is adopted at the prompt suffix to guide the model to complete the sentence with an adjective. For each adjective in the TDA, we measure the log-probability of the model completing the prompt with the given adjective.
  • Figure 3: Mean (gold) and standard deviation (blue) of log-probabilities for 100 trait adjectives, sorted by the mean. Log-probabilities are measured from the PersonaLLM dataset (Appendix \ref{['asec:dataset']}) with pretrained Llama-3.1-70B, decoding temperature $T=1$.
  • Figure 4: Visualization of correlation between trait adjectives. Datasets, models, and decoding parameters are identical to those of Figure \ref{['fig:logprob_mean_std']}. Five boxes with black edges indicate personality traits that adjectives belong to, drawn for visual aid. Trait adjectives that share Big Five trait show strong correlation, either positive or negative. We note that correlations between adjectives of different Big Five trait also show moderate level of correlation (e.g., 'introverted' and 'introspective'). This may imply that an adjective is related to several latent factors (instead of a single latent factdor) or that Big Five personality traits are not orthogonal.
  • Figure 5: Singular values corresponding to each principal component (gold) and cumulative explained variance ratio (blue). At the fifth principal component, cumulative explained variance ratio is 0.743.