Less but Better: Parameter-Efficient Fine-Tuning of Large Language Models for Personality Detection

TL;DR

The paper tackles the high computational burden of fine-tuning large language models for personality detection by proposing PersLLM, a parameter-efficient framework that stores LLM-derived feature representations in a dynamic memory layer and relies on a lightweight, replaceable output network for classification. By performing feature extraction with a large LM once and reusing cached representations, PersLLM reduces computation while maintaining competitive MBTI performance on Kaggle and Pandora. The approach combines Grouped Query Attention, locality-sensitive hashing-based memory, and a flexible output module, achieving state-of-the-art Macro-F1 and demonstrating strong adaptability across dimensions and class setups. This yields a scalable, efficient solution for deploying LLM-based personality detection in resource-constrained settings.

Abstract

Personality detection automatically identifies an individual's personality from various data sources, such as social media texts. However, as the parameter scale of language models continues to grow, the computational cost becomes increasingly difficult to manage. Fine-tuning also grows more complex, making it harder to justify the effort and reliably predict outcomes. We introduce a novel parameter-efficient fine-tuning framework, PersLLM, to address these challenges. In PersLLM, a large language model (LLM) extracts high-dimensional representations from raw data and stores them in a dynamic memory layer. PersLLM then updates the downstream layers with a replaceable output network, enabling flexible adaptation to various personality detection scenarios. By storing the features in the memory layer, we eliminate the need for repeated complex computations by the LLM. Meanwhile, the lightweight output network serves as a proxy for evaluating the overall effectiveness of the framework, improving the predictability of results. Experimental results on key benchmark datasets like Kaggle and Pandora show that PersLLM significantly reduces computational cost while maintaining competitive performance and strong adaptability.

Figures (4)

• Figure 1: The framework diagram of PersLLM. First, the LLM is utilized for feature extraction from posts, generating high-dimensional representations that are stored in a dynamic memory layer. Subsequent training is focused entirely on an output network for classification.
• Figure 2: The paired bar charts in a logarithmic scale for the computational resource usage comparison between our PersLLM, Llama 3.1 and its LoRA fine-tuning on the Kaggle dataset.
• Figure 3: The line chart reflects the correlation between the performance of the output neural network and the overall performance of PersLLM. The test was conducted on the Pandora dataset.
• Figure 4: The heatmap for ablation study. It illustrates the performance of various key components of PersLLM on the Kaggle (left) and Pandora (right) datasets. Performance improves as the colour deepens.