Evaluating and Inducing Personality in Pre-trained Language Models

TL;DR

This work introduces the Machine Personality Inventory (MPI), a Big Five–based, zero-shot questionnaire to quantify personality-like behaviors in large language models. It demonstrates that several aligned LLMs exhibit human-like stability and distribution in OCEAN traits and presents Personality Prompting (P^2), a prompting chain to induce specific personality profiles, validated via MPI and vignette tests with human raters. The study shows that induced personalities can be controlled and verified across inventories and real-world scenarios, highlighting potential for tailored interactions and deeper behavioral understanding of LLMs. It also discusses limitations, safety implications, and biases, calling for broader exploration of machine personality in responsible AI research.

Abstract

Standardized and quantified evaluation of machine behaviors is a crux of understanding LLMs. In this study, we draw inspiration from psychometric studies by leveraging human personality theory as a tool for studying machine behaviors. Originating as a philosophical quest for human behaviors, the study of personality delves into how individuals differ in thinking, feeling, and behaving. Toward building and understanding human-like social machines, we are motivated to ask: Can we assess machine behaviors by leveraging human psychometric tests in a principled and quantitative manner? If so, can we induce a specific personality in LLMs? To answer these questions, we introduce the Machine Personality Inventory (MPI) tool for studying machine behaviors; MPI follows standardized personality tests, built upon the Big Five Personality Factors (Big Five) theory and personality assessment inventories. By systematically evaluating LLMs with MPI, we provide the first piece of evidence demonstrating the efficacy of MPI in studying LLMs behaviors. We further devise a Personality Prompting (P^2) method to induce LLMs with specific personalities in a controllable way, capable of producing diverse and verifiable behaviors. We hope this work sheds light on future studies by adopting personality as the essential indicator for various downstream tasks, and could further motivate research into equally intriguing human-like machine behaviors.

Figures (2)

• Figure 1: Evaluating and inducing personality in llm.llm are trained on multitudinous textual corpora and have the potential to exhibit various personalities. We evaluate llm' personality using our mpi and further introduce a prompting-based method to induce llm with a certain personality in a controllable manner. OCEAN refers to five key factors: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.
• Figure 2: Control via method. An example of Extraversion control via our method. Given a specific dimension in big5, a naive prompt employs an intuitive template. Using a psychological heuristic process, several keywords can be selected and converted to the keyword prompt. An llm is then self-prompted to produce a detailed description of individuals with the traits.