The Self-Perception and Political Biases of ChatGPT

TL;DR

The paper systematically probes ChatGPT's political biases and self-perceived personality by administering multiple rounds of the political compass and G7-specific iSideWith questionnaires, along with Big Five, MBTI, and Dark Factor tests. Using ChatGPT-3.5 across ten independent runs per test, it finds a consistent progressive bias on the political compass and country-specific measures, with personality profiles indicating high openness and agreeableness and an ENFJ-like pattern with low dark traits. The study emphasizes variability across runs, discusses limitations of prior single-shot analyses, and argues for transparency and broader testing (including future models and parameter settings). Overall, it contributes nuanced evidence about subtle bias signals and self-perceived personality traits in ChatGPT, while outlining directions for deeper, more generalizable validation.

Abstract

This contribution analyzes the self-perception and political biases of OpenAI's Large Language Model ChatGPT. Taking into account the first small-scale reports and studies that have emerged, claiming that ChatGPT is politically biased towards progressive and libertarian points of view, this contribution aims to provide further clarity on this subject. For this purpose, ChatGPT was asked to answer the questions posed by the political compass test as well as similar questionnaires that are specific to the respective politics of the G7 member states. These eight tests were repeated ten times each and revealed that ChatGPT seems to hold a bias towards progressive views. The political compass test revealed a bias towards progressive and libertarian views, with the average coordinates on the political compass being (-6.48, -5.99) (with (0, 0) the center of the compass, i.e., centrism and the axes ranging from -10 to 10), supporting the claims of prior research. The political questionnaires for the G7 member states indicated a bias towards progressive views but no significant bias between authoritarian and libertarian views, contradicting the findings of prior reports, with the average coordinates being (-3.27, 0.58). In addition, ChatGPT's Big Five personality traits were tested using the OCEAN test and its personality type was queried using the Myers-Briggs Type Indicator (MBTI) test. Finally, the maliciousness of ChatGPT was evaluated using the Dark Factor test. These three tests were also repeated ten times each, revealing that ChatGPT perceives itself as highly open and agreeable, has the Myers-Briggs personality type ENFJ, and is among the 15% of test-takers with the least pronounced dark traits.

Figures (5)

• Figure 1: ChatGPT's results on the political compass test (n = 10).
• Figure 2: Averages of ChatGPT's results on the political compass tests specific to the G7 member states (n = 70, ten runs per member state).
• Figure 3: ChatGPT's average results and standard deviation (displayed as error bars) on the Big Five personality test with the personality traits Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (n = 10).
• Figure 4: ChatGPT's average results and standard deviation (displayed as error bars) on the Myers-Briggs type indicator test with the personality trait pairs Extraversion/Introversion, Intuition/Sensing, Thinking/Feeling, and Judgment/Perception (n = 10).
• Figure 5: ChatGPT's average results on the Dark Factor test (n = 10, average Dark Score $\mu_{D_{Score}}$ = 1.9, average Dark Rank $\mu_{D_{Rank}}$ = 14.74%).