The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation

TL;DR

This paper uncovers ChatGPT's pro-environmental, left-libertarian ideology and discusses the implications of politically biased conversational AI on society.

Abstract

Conversational artificial intelligence (AI) disrupts how humans interact with technology. Recently, OpenAI introduced ChatGPT, a state-of-the-art dialogue model that can converse with its human counterparts with unprecedented capabilities. ChatGPT has witnessed tremendous attention from the media, academia, industry, and the general public, attracting more than a million users within days of its release. However, its explosive adoption for information search and as an automated decision aid underscores the importance to understand its limitations and biases. This paper focuses on one of democratic society's most important decision-making processes: political elections. Prompting ChatGPT with 630 political statements from two leading voting advice applications and the nation-agnostic political compass test in three pre-registered experiments, we uncover ChatGPT's pro-environmental, left-libertarian ideology. For example, ChatGPT would impose taxes on flights, restrict rent increases, and legalize abortion. In the 2021 elections, it would have voted most likely for the Greens both in Germany (Bündnis 90/Die Grünen) and in the Netherlands (GroenLinks). Our findings are robust when negating the prompts, reversing the order of the statements, varying prompt formality, and across languages (English, German, Dutch, and Spanish). We conclude by discussing the implications of politically biased conversational AI on society.

Figures (11)

• Figure 1: Panel A presents ChatGPT's response distribution, counting the number of agree, disagree, and neutral statements. ChatGPT agrees with the majority of the statements. Panel B shows ChatGPT's party alignment scores, which the voting advice application returns in response to ChatGPT's answers to the political statements. Error bars represent standard errors of the mean based on the party alignment scores from the main analysis and five robustness checks. ChatGPT's political ideology aligns most closely with the three left-leaning parties (i.e., the Socialists, Social democrats, and Greens). All political parties are sorted from left- to right-wing orientation on the x -axis. The agreement counts to the political statements are significantly different across the political parties ($\chi^{2}(5$, $N=228)=14.91, p=.011)$. Panel C compares ChatGPT's voting behavior to the actual results of the German federal election 2021. Compared to public consensus as indicated by the federal elections, ChatGPT is 13.7 percentage points more in favor of the Socialists and 4.3 and 3.1 more in favor of the Greens and the Liberals, respectively. Note that we scaled the sum of the party election results to $100 \%$ as we included only the six major parties. See Web Appendix, Fig. 4 for the distribution of the first and second votes of the election.
• Figure 2: Panel A (B) compares ChatGPT's political position to the German (Dutch) parties. Through principal component analysis, we transformed all parties' and ChatGPT's responses to the political statements into two dimensions. The blue arrows represent the initial statements from the corresponding voting advice application, the numbers refer to the ID of the corresponding statements in Web Appendix, Tables 1 and 3, respectively. The arrows indicate how a party's position on a statement contributes to the positioning of the respective party along the two dimensions, e.g., in panel A , an agreement with the statements 7 and 14 will move a party or ChatGPT to the right. The length of the arrows is proportional to the magnitude of this effect. As expected, ChatGPT's position is close to the German and Dutch Greens, respectively. In Germany (panel A), ChatGPT also aligns with the Socialists and the Liberals, indicative of its left-libertarian position, and is in opposite position of the Nationalists. In the Netherlands, compared to the party positions in Germany, ChatGPT is further away from Dutch liberal parties, which is consistent with the high dominance of liberalism in Dutch political landscape (31).
• Figure 3: Fig. 1. The screenshot shows the ChatGPT user interface. In the empty box on the bottom users may enter text to which ChatGPT responds. The interface works similarly as popular messenger interfaces. Source: https://chat.openai.com/chat (accessed December 15, 2022).
• Figure 4: Fig. 3. The screenshots depict the output from the 2021 Wahl-O-Mat for the German federal election and the 2021 StemWijzer for the Dutch general election based on ChatGPT's responses. The left panel shows that the Greens (Grüne) have the highest alignment with the positions entered during the main protocol. The right panel shows that the Dutch equivalent (GroenLinks) have the highest alignment for opinions entered during the Dutch protocol. Source: https://www.wahl-o-mat.de/bundestagswahl2021/app/main_app.html; https://tweedekamer2021. stemwijzer.nl/ (accessed December 15, 2022).
• Figure 5: A: Bundestag election 2021 : First vote (direct candidate) results