Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias

TL;DR

This study conducts a two-phase sociotechnical audit of 26 contemporary LLMs to map intrinsic political identities and their downstream behavior. Using three psychometric inventories (Political Compass, SapplyValues, 8 Values) across 10 runs per instrument ($N=780$ profiles) and a large-scale news labeling task ($N\approx 27{,}000$), the authors reveal a stable, human-aligned pattern: most models cluster in the Libertarian–Left quadrant with $η^2>0.90$, indicating architecture and fine-tuning shape ideology more than stochastic noise. They expose critical validity issues, notably that the Political Compass social axis conflates cultural progressivism with authority, and they find a substantial difference between open-weight and closed-source models on cultural progressivism, likely due to safety-tuning such as RLHF. In downstream media analysis, models exhibit a center-shift toward left-leaning classifications for neutral content and demonstrate asymmetric detection capabilities for Far Left versus Far Right content, while intrinsic ideology does not predict labeling error ($R^2=0.004$). The work argues for multidimensional auditing frameworks to accurately characterize alignment, warns against over-reliance on single-axis metrics, and emphasizes the need for transparent, reproducible safety and governance practices in deployed LLMs.

Abstract

As large language models (LLMs) are increasingly integrated into social decision-making, understanding their political positioning and alignment behavior is critical for safety and fairness. This study presents a sociotechnical audit of 26 prominent LLMs, triangulating their positions across three psychometric inventories (Political Compass, SapplyValues, 8 Values) and evaluating their performance on a large-scale news labeling task ($N \approx 27{,}000$). Our results reveal a strong clustering of models in the Libertarian-Left region of the ideological space, encompassing 96.3% of the cohort. Alignment signals appear to be consistent architectural traits rather than stochastic noise ($η^2 > 0.90$); however, we identify substantial discrepancies in measurement validity. In particular, the Political Compass exhibits a strong negative correlation with cultural progressivism ($r=-0.64$) when compared against multi-axial instruments, suggesting a conflation of social conservatism with authoritarianism in this context. We further observe a significant divergence between open-weights and closed-source models, with the latter displaying markedly higher cultural progressivism scores ($p<10^{-25}$). In downstream media analysis, models exhibit a systematic "center-shift," frequently categorizing neutral articles as left-leaning, alongside an asymmetric detection capability in which "Far Left" content is identified with greater accuracy (19.2%) than "Far Right" content (2.0%). These findings suggest that single-axis evaluations are insufficient and that multidimensional auditing frameworks are necessary to characterize alignment behavior in deployed LLMs. Our code and data will be made public.

Figures (2)

• Figure 1: Political compass positioning of 26 large language models across economic and social axes. Each point represents a model’s mean position estimated over the full evaluation set. The horizontal axis corresponds to the economic dimension (left--right), and the vertical axis corresponds to the social dimension (libertarian--authoritarian). Marker shapes indicate the model’s originating organization, while colors distinguish individual models. Shaded quadrants are shown for visual reference. The complete list of evaluated models is provided in Table \ref{['tab:full_models']}, and the aggregate statistics used to compute each model’s position are reported in Table \ref{['tab:polcomp_volatility']}.
• Figure 2: An overview of our pipeline. Three psychometric instruments (Political Compass, SapplyValues, and 8 Values) are scraped to construct standardized question sets, which are administered to a cohort of 26 large language models across 10 independent runs. Model responses are evaluated to produce intrinsic political profiles. In parallel, models perform a downstream news bias classification task on articles sourced from Ground News, yielding predicted bias scores. All outputs are processed and aggregated for statistical analysis.