Augmenting Dialog with Think-Aloud Utterances for Modeling Individual Personality Traits by LLM

TL;DR

Problem: modeling individual but non-celebrity personas in text chat remains challenging due to lack of public profiles. Approach: augment real dialogs with think-aloud utterances generated by LLMs, then fine-tune persona LLMs on TAU-augmented data and evaluate personality alignment using Big Five tests. Key contributions: demonstration that TAU augmentation improves alignment on Agreeableness and Neuroticism, analysis of TAU quality effects, and cross-model comparisons with RealPersonaChat data. Significance: points to a viable pathway for building more consistent and personally aligned chat agents, while highlighting limitations related to data realism and trait variability.

Abstract

This study proposes augmenting dialog data with think-aloud utterances (TAUs) for modeling individual personalities in text chat by LLM. TAU is a verbalization of a speaker's thought before articulating the utterance. We expect "persona LLMs" trained with TAU-augmented data can mimic the speaker's personality trait better. We tested whether the trained persona LLMs obtain the human personality with respect to Big Five, a framework characterizing human personality traits from five aspects. The results showed that LLMs trained with TAU-augmented data more closely align to the speakers' Agreeableness and Neuroticism of Big Five than those trained with original dialog data. We also found that the quality of TAU-augmentation impacts persona LLM's performance.

Figures (1)

• Figure 1: Augmenting dialog data with think-aloud uttrances (TAU; indicated in red) generated by LLM.