Revealing Personality Traits: A New Benchmark Dataset for Explainable Personality Recognition on Dialogues

TL;DR

This work targets explainable personality recognition from dialogues by introducing the Chain-of-Personality-Evidence (CoPE) framework, which traces the reasoning from immediate contextual states to short-term states and ultimately to long-term traits. Building on this, the authors create PersonalityEvd, a Chinese-dialogue dataset with detailed state and trait evidence designed for two tasks: Evidence grounded Personality State Recognition (EPR-S) and Evidence grounded Personality Trait Recognition (EPR-T). They benchmark several large language models, showing that while models can partially infer trait information, providing coherent, evidence-grounded explanations remains challenging. Human evaluation confirms the high quality of the dataset while highlighting gaps in current model explanations, underscoring the need for further research into explainable, evidence-based personality inference in dialogues.

Abstract

Personality recognition aims to identify the personality traits implied in user data such as dialogues and social media posts. Current research predominantly treats personality recognition as a classification task, failing to reveal the supporting evidence for the recognized personality. In this paper, we propose a novel task named Explainable Personality Recognition, aiming to reveal the reasoning process as supporting evidence of the personality trait. Inspired by personality theories, personality traits are made up of stable patterns of personality state, where the states are short-term characteristic patterns of thoughts, feelings, and behaviors in a concrete situation at a specific moment in time. We propose an explainable personality recognition framework called Chain-of-Personality-Evidence (CoPE), which involves a reasoning process from specific contexts to short-term personality states to long-term personality traits. Furthermore, based on the CoPE framework, we construct an explainable personality recognition dataset from dialogues, PersonalityEvd. We introduce two explainable personality state recognition and explainable personality trait recognition tasks, which require models to recognize the personality state and trait labels and their corresponding support evidence. Our extensive experiments based on Large Language Models on the two tasks show that revealing personality traits is very challenging and we present some insights for future research. Our data and code are available at https://github.com/Lei-Sun-RUC/PersonalityEvd.

Figures (8)

• Figure 1: Chain-of-Personality-Evidence (CoPE) framework illustrates the reasoning process for revealing supporting evidence of personality traits.
• Figure 2: A Speaker's explainable personality annotation of the Neuroticism dimension from the Big-Five Personality Model, including dialogue-level personality state and speaker-level personality trait labels with corresponding evidence (natural language reasoning process). U# denotes evidence utterances and D# denotes evidence dialogues. The neuroticism dimension contains anxiety, depression, and emotional volatility three facets in the BFI-2 scale. For personality trait evidence annotation, we annotate the natural language reasoning process for each facet.
• Figure 3: (a) The distribution of state labels. (b) The distribution of trait labels. (c) The ratio of the state label different from the trait label. (O: openness, C: conscientiousness, E: extraversion, A: agreeableness, N: neuroticism)
• Figure 4: The word clouds of personality states reasoning process on openness and conscientiousness dimensions.
• Figure 5: Facets and Items of The BFI-2 Scale. (Dim: dimension)