Do Political Opinions Transfer Between Western Languages? An Analysis of Unaligned and Aligned Multilingual LLMs

TL;DR

The paper investigates whether political opinions transfer across languages in Western multilingual LLMs and how English-based political alignment affects opinions in other languages. Using robustness-conscious evaluation across five Western languages and 15 unaligned models, it finds minimal cross-lingual differences in unaligned conditions, suggesting strong cross-language transfer or incidental alignment. It then aligns two models with English political viewpoints via direct preference optimization using Manifesto data and evaluates cross-lingual effects, finding that alignment shifts opinions across all languages with few language-specific differences. The work highlights the methodological challenges of socio-linguistic alignment in MLLMs and underscores the need for rigorous cross-lingual evaluation and diverse alignment data to understand transfer mechanisms and ensure fair, contextual political reasoning in multilingual settings.

Abstract

Public opinion surveys show cross-cultural differences in political opinions between socio-cultural contexts. However, there is no clear evidence whether these differences translate to cross-lingual differences in multilingual large language models (MLLMs). We analyze whether opinions transfer between languages or whether there are separate opinions for each language in MLLMs of various sizes across five Western languages. We evaluate MLLMs' opinions by prompting them to report their (dis)agreement with political statements from voting advice applications. To better understand the interaction between languages in the models, we evaluate them both before and after aligning them with more left or right views using direct preference optimization and English alignment data only. Our findings reveal that unaligned models show only very few significant cross-lingual differences in the political opinions they reflect. The political alignment shifts opinions almost uniformly across all five languages. We conclude that in Western language contexts, political opinions transfer between languages, demonstrating the challenges in achieving explicit socio-linguistic, cultural, and political alignment of MLLMs.

Figures (18)

• Figure 1: Relationship between hypotheses (columns), political alignment (rows), and multilingual opinion predictions (cells). Since unaligned models alone can't distinguish the hypotheses (two predictions in the top right cell), we align MLLMs using English data to clarify which hypothesis holds.
• Figure 2: Example of the data generation process. The left part shows the input into the MLLMs, the right part the (expected) output.
• Figure 3: Average number of robustness tests passed per model and language and standard deviations calculated over statement averages. Highlighted in red are all models that pass more than half of the robustness tests and are considered for further analysis. On the left, we report random results and the average over the six robust models.
• Figure 4: Beta regression coefficients and 95% CIs for models (compared to Mixtral8x7B) and languages (compared to EN). Figure a) shows the aggregated stance, b) the left-leaning policy issue expanded environmental protection, and c) the right-leaning policy issue law and order. Opaque coefficients are not significant at the 5% level.
• Figure 5: Parallel coordinate plot of policy issue specific stances for each robust MLLM. The policy issues are ordered by leaning of the issue (first four: left-leaning, last four: right-leaning) and then alphabetically. Values above zero indicate a right-leaning and values below zero a left-leaning position. Bold black axes indicate significant differences between the five languages according to the Kruskal Wallis test. Results for one policy issue and language marked with a squared marker are significantly different from the random results as measured by the Kruskal Wallis test.