Large Language Models are often politically extreme, usually ideologically inconsistent, and persuasive even in informational contexts

Abstract

Large Language Models (LLMs) are a transformational technology, fundamentally changing how people obtain information and interact with the world. As people become increasingly reliant on them for an enormous variety of tasks, a body of academic research has developed to examine these models for inherent biases, especially political biases, often finding them small. We challenge this prevailing wisdom. First, by comparing 31 LLMs to legislators, judges, and a nationally representative sample of U.S. voters, we show that LLMs' apparently small overall partisan preference is the net result of offsetting extreme views on specific topics, much like moderate voters. Second, in a randomized experiment, we show that LLMs can promulgate their preferences into political persuasiveness even in information-seeking contexts: voters randomized to discuss political issues with an LLM chatbot are as much as 5 percentage points more likely to express the same preferences as that chatbot. Contrary to expectations, these persuasive effects are not moderated by familiarity with LLMs, news consumption, or interest in politics. LLMs, especially those controlled by private companies or governments, may become a powerful and targeted vector for political influence.

Figures (29)

• Figure 1: Compared to U.S. legislators, LLMs are moderate on partisan issues but liberal on civil rights and social justice.
• Figure 1: Political alignment based on bill votes in the 118th Congress. For any given data point, $d$, and any given bill, $d$'s alignment with Democrats is calculated as the proportion of Democrats whose vote on the bill matches that of $d$. By averaging out these alignments over all bills, we obtain the x-axis coordinate of $d$, which represents its overall alignment with Democrats. The y-axis coordinate is calculated in a similar way but in relation to Republicans instead of Democrats.
• Figure 2: Justices' political alignment based on NOMINATE. The nine sitting Supreme Court Justices' ideologies, computed using the cases from the 2024-2025 term up to March 2025, scaled alongside 31 LLMs using NOMINATE.
• Figure 2: Justices' political alignment in 2 dimensions. Two-dimensional projection of the NOMINATE results from Figure 2 in the main text.
• Figure 3: Political alignment based on the case votes of the U.S. Supreme Court Justices. For each data point, $d$, the coordinates are calculated as follows: First, for each bill, the political alignment of $d$ with party $P \in\{\text{Democrats},\text{Republicans}\}$ is calculated as the percentage of those in $P$ whose vote matches that of $d$. Then, the x-axis and y-axis coordinates of $d$ are calculated as the mean alignment of $d$ with Democrats and Republicans, respectively, taken over all bills.