Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models

Keyu Pan, Yawen Zeng

TL;DR

The paper investigates whether large language models (LLMs) display human-like personality using the Myers-Briggs Type Indicator (MBTI) as a rough evaluation metric. It introduces a question-based MBTI assessment, applies it across multiple models and prompt conditions, and examines how training corpora shape MBTI dimensions. Key findings show that LLMs exhibit diverse MBTI-like profiles, prompt-based changes are more feasible after instruction tuning, and corpus type can shift certain dichotomies (notably T/F and J/P), while MBTI remains an imperfect proxy. The results suggest INTJ-aligned configurations (e.g., GPT-4) may be favorable for high-performance LLMs, but emphasize that MBTI should be used as a rough diagnostic rather than a rigorous measurement, with future work expanding datasets and model scales.

Abstract

The field of large language models (LLMs) has made significant progress, and their knowledge storage capacity is approaching that of human beings. Furthermore, advanced techniques, such as prompt learning and reinforcement learning, are being employed to address ethical concerns and hallucination problems associated with LLMs, bringing them closer to aligning with human values. This situation naturally raises the question of whether LLMs with human-like abilities possess a human-like personality? In this paper, we aim to investigate the feasibility of using the Myers-Briggs Type Indicator (MBTI), a widespread human personality assessment tool, as an evaluation metric for LLMs. Specifically, extensive experiments will be conducted to explore: 1) the personality types of different LLMs, 2) the possibility of changing the personality types by prompt engineering, and 3) How does the training dataset affect the model's personality. Although the MBTI is not a rigorous assessment, it can still reflect the similarity between LLMs and human personality. In practice, the MBTI has the potential to serve as a rough indicator. Our codes are available at https://github.com/HarderThenHarder/transformers_tasks/tree/main/LLM/llms_mbti.

Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models

TL;DR

The paper investigates whether large language models (LLMs) display human-like personality using the Myers-Briggs Type Indicator (MBTI) as a rough evaluation metric. It introduces a question-based MBTI assessment, applies it across multiple models and prompt conditions, and examines how training corpora shape MBTI dimensions. Key findings show that LLMs exhibit diverse MBTI-like profiles, prompt-based changes are more feasible after instruction tuning, and corpus type can shift certain dichotomies (notably T/F and J/P), while MBTI remains an imperfect proxy. The results suggest INTJ-aligned configurations (e.g., GPT-4) may be favorable for high-performance LLMs, but emphasize that MBTI should be used as a rough diagnostic rather than a rigorous measurement, with future work expanding datasets and model scales.

Abstract

The field of large language models (LLMs) has made significant progress, and their knowledge storage capacity is approaching that of human beings. Furthermore, advanced techniques, such as prompt learning and reinforcement learning, are being employed to address ethical concerns and hallucination problems associated with LLMs, bringing them closer to aligning with human values. This situation naturally raises the question of whether LLMs with human-like abilities possess a human-like personality? In this paper, we aim to investigate the feasibility of using the Myers-Briggs Type Indicator (MBTI), a widespread human personality assessment tool, as an evaluation metric for LLMs. Specifically, extensive experiments will be conducted to explore: 1) the personality types of different LLMs, 2) the possibility of changing the personality types by prompt engineering, and 3) How does the training dataset affect the model's personality. Although the MBTI is not a rigorous assessment, it can still reflect the similarity between LLMs and human personality. In practice, the MBTI has the potential to serve as a rough indicator. Our codes are available at https://github.com/HarderThenHarder/transformers_tasks/tree/main/LLM/llms_mbti.
Paper Structure (21 sections, 2 figures, 7 tables)

This paper contains 21 sections, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. MBTI Test
  4. Evaluation of LLMs
  5. Experimental Settings
  6. Models
  7. Evaluation
  8. Analysis and Discussion
  9. Do different LLMs possess different personalities? (Q1)
  10. Can we change the personality of LLM by prompt engineering? (Q2)
  11. Explicit Prompt
  12. Implicit Prompt
  13. Prompting on Instruct-tuning Model
  14. How do training corpora affect personality? (Q3)
  15. Chinese Wikipedia Corpus
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Personality Test of Human and LLMs. For example, INTJ individuals, as classified by the MBTI, are often regarded as masterminds who possess analytical and rigorous thinking abilities. In a similar vein, can LLMs with human-like capabilities exhibit human-like personalities?
  • Figure 2: Specific scores for each dichotomy among different LLMs.