Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models

Ivar Frisch, Mario Giulianelli

TL;DR

This paper investigates whether LLM agents conditioned on personality prompts maintain their assigned traits during interaction and whether they linguistically align to conversational partners. Using GPT-3.5-turbo, the authors bootstrap a population of agents and assign Creative or Analytical Big Five-inspired personas, evaluating explicit personality via the Big Five Inventory and implicit alignment via LIWC analysis in non-interactive and interactive tasks. Results show that personality consistency is higher for the Creative profile, while interaction yields partial linguistic alignment and asymmetric adaptation, with analytic agents more prone to drift toward creativity. The findings highlight the need for robust persona-conditioning methods to support human-like, interactive LLM environments and lay groundwork for future studies on multi-turn inter-agent dialogue and alignment metrics.

Abstract

While both agent interaction and personalisation are vibrant topics in research on large language models (LLMs), there has been limited focus on the effect of language interaction on the behaviour of persona-conditioned LLM agents. Such an endeavour is important to ensure that agents remain consistent to their assigned traits yet are able to engage in open, naturalistic dialogues. In our experiments, we condition GPT-3.5 on personality profiles through prompting and create a two-group population of LLM agents using a simple variability-inducing sampling algorithm. We then administer personality tests and submit the agents to a collaborative writing task, finding that different profiles exhibit different degrees of personality consistency and linguistic alignment to their conversational partners. Our study seeks to lay the groundwork for better understanding of dialogue-based interaction between LLMs and highlights the need for new approaches to crafting robust, more human-like LLM personas for interactive environments.

LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models

TL;DR

This paper investigates whether LLM agents conditioned on personality prompts maintain their assigned traits during interaction and whether they linguistically align to conversational partners. Using GPT-3.5-turbo, the authors bootstrap a population of agents and assign Creative or Analytical Big Five-inspired personas, evaluating explicit personality via the Big Five Inventory and implicit alignment via LIWC analysis in non-interactive and interactive tasks. Results show that personality consistency is higher for the Creative profile, while interaction yields partial linguistic alignment and asymmetric adaptation, with analytic agents more prone to drift toward creativity. The findings highlight the need for robust persona-conditioning methods to support human-like, interactive LLM environments and lay groundwork for future studies on multi-turn inter-agent dialogue and alignment metrics.

Abstract

While both agent interaction and personalisation are vibrant topics in research on large language models (LLMs), there has been limited focus on the effect of language interaction on the behaviour of persona-conditioned LLM agents. Such an endeavour is important to ensure that agents remain consistent to their assigned traits yet are able to engage in open, naturalistic dialogues. In our experiments, we condition GPT-3.5 on personality profiles through prompting and create a two-group population of LLM agents using a simple variability-inducing sampling algorithm. We then administer personality tests and submit the agents to a collaborative writing task, finding that different profiles exhibit different degrees of personality consistency and linguistic alignment to their conversational partners. Our study seeks to lay the groundwork for better understanding of dialogue-based interaction between LLMs and highlights the need for new approaches to crafting robust, more human-like LLM personas for interactive environments.
Paper Structure (24 sections, 4 figures, 7 tables)

This paper contains 24 sections, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Experimental Approach
  3. Population Bootstrapping
  4. Personality-Conditioned LLM Agents
  5. Explicit Personality Assessment
  6. Implicit Personality Assessment
  7. Results
  8. Experiment 1: Non-Interactive Condition
  9. Are the assigned personality traits reflected in responses to the BFI test?
  10. Are the assigned personality traits reflected in LLM agents' language use?
  11. Experiment 2: Interactive Condition
  12. Do LLM agents' responses to BFI tests change as a result of interaction?
  13. Do agents exhibit linguistic alignment to their conversational partners?
  14. Conclusion
  15. Prompts
  16. ...and 9 more sections

Figures (4)

  • Figure 1: BFI scores of personality-conditioned LLM agents before (a) and after (b) the non-interactive writing task.
  • Figure 2: Language use in the non-interactive vs. interactive condition. Left (a, b): 2D visualisation, through PCA, of LIWC vectors obtained from the generated stories. Each point represents the language use of a single agent. Right (c, d): Point-biserial correlation coefficients between the top 5 LIWC features and personality profiles. Positive coefficients indicate correlation with creative group, negative coefficients with the analytic group.
  • Figure 3: Distribution of top 5 Spearman correlation coefficients per personality trait.
  • Figure 4: BFI scores of personality-conditioned LLM after the interactive writing task.