From Classification to Ranking: Enhancing LLM Reasoning Capabilities for MBTI Personality Detection

TL;DR

The paper reframes MBTI personality detection from a four-dimension classification task to a list-wise ranking problem using social media posts. It proposes a two-stage LLM post-training pipeline, combining supervised fine-tuning with reasoning traces distilled from a teacher model and Group Relative Policy Optimization (GRPO) guided by a ranking-based $NDCG$ reward and a dimension-similarity reward. The approach achieves state-of-the-art results on Kaggle and PANDORA datasets, improving both binary and multiclass MBTI predictions by capturing inter-dimension interactions and reducing reliance on expert prompts. This ranking-centric framework enables autonomous pattern learning from behavioral data and offers robust generalization advantages for practical personality assessment tasks.

Abstract

Personality detection aims to measure an individual's corresponding personality traits through their social media posts. The advancements in Large Language Models (LLMs) offer novel perspectives for personality detection tasks. Existing approaches enhance personality trait analysis by leveraging LLMs to extract semantic information from textual posts as prompts, followed by training classifiers for categorization. However, accurately classifying personality traits remains challenging due to the inherent complexity of human personality and subtle inter-trait distinctions. Moreover, prompt-based methods often exhibit excessive dependency on expert-crafted knowledge without autonomous pattern-learning capacity. To address these limitations, we view personality detection as a ranking task rather than a classification and propose a corresponding reinforcement learning training paradigm. First, we employ supervised fine-tuning (SFT) to establish personality trait ranking capabilities while enforcing standardized output formats, creating a robust initialization. Subsequently, we introduce Group Relative Policy Optimization (GRPO) with a specialized ranking-based reward function. Unlike verification tasks with definitive solutions, personality assessment involves subjective interpretations and blurred boundaries between trait categories. Our reward function explicitly addresses this challenge by training LLMs to learn optimal answer rankings. Comprehensive experiments have demonstrated that our method achieves state-of-the-art performance across multiple personality detection benchmarks.

Figures (4)

• Figure 1: A framework for LLM-based MBTI personality detection. The framework leverages a single user's social media posts as input to corresponding personality traits along four binary dimensions of Introversion vs. Extroversion, Sensing vs. iNtuition, Think vs. Feeling, and Perception vs. Judging.
• Figure 2: An overview of our PerDet R1, the two-stage LLM post-training workflow. In the first stage, we adopt Qwen-plus as the teacher model to generate a reasoning trace for comparing different MBTI types. Subsequently, the base model learning formatted data via SFT to get a reasoning model. In the second stage, the reasoning model optimizes via Group Relative Policy Optimization (GRPO) with NDCG and dimension similarity reward function to enhance ranking capability.
• Figure 3: An example of prompts and generated reasoning trace with think and answer label for top-k MBTI types ranking.
• Figure 4: NDCG Reward (smoothed with 0.8 sigma) comparison during RL training stage on Kaggle Dataset.