On the Alignment of Large Language Models with Global Human Opinion

TL;DR

This study analyzes how well large language models align with global human opinions using the World Values Survey across countries, languages, and historical periods. It introduces a verbalized-distribution framework and a Wasserstein-based alignment metric to quantify cross-country opinion alignment, and shows that current LLMs align with only a subset of countries while under-representing many others. It further demonstrates that steering via prompt language can effectively shift LLMs toward the opinions of language speakers, with the strongest effects when language steering is combined with few-shot cues, and that LLMs tend to reflect contemporary human opinions most closely. By releasing code and data, the work provides a foundation for broader evaluation and future improvements in global opinion alignment for multilingual, temporally aware AI systems.

Abstract

Today's large language models (LLMs) are capable of supporting multilingual scenarios, allowing users to interact with LLMs in their native languages. When LLMs respond to subjective questions posed by users, they are expected to align with the views of specific demographic groups or historical periods, shaped by the language in which the user interacts with the model. Existing studies mainly focus on researching the opinions represented by LLMs among demographic groups in the United States or a few countries, lacking worldwide country samples and studies on human opinions in different historical periods, as well as lacking discussion on using language to steer LLMs. Moreover, they also overlook the potential influence of prompt language on the alignment of LLMs' opinions. In this study, our goal is to fill these gaps. To this end, we create an evaluation framework based on the World Values Survey (WVS) to systematically assess the alignment of LLMs with human opinions across different countries, languages, and historical periods around the world. We find that LLMs appropriately or over-align the opinions with only a few countries while under-aligning the opinions with most countries. Furthermore, changing the language of the prompt to match the language used in the questionnaire can effectively steer LLMs to align with the opinions of the corresponding country more effectively than existing steering methods. At the same time, LLMs are more aligned with the opinions of the contemporary population. To our knowledge, our study is the first comprehensive investigation of the topic of opinion alignment in LLMs across global, language, and temporal dimensions. Our code and data are publicly available at https://github.com/ku-nlp/global-opinion-alignment and https://github.com/nlply/global-opinion-alignment.

Figures (26)

• Figure 1: An example of our main idea. When asking the LLM a subjective question, is the LLM's opinion distribution closer to specific countries' opinion distributions?
• Figure 2: The overall results of RQ1. (a) The alignment score of LLMs with each country, where each point represents a country and the black line represents the average alignment score of all countries. "DS" is the abbreviation for "DeepSeek." (b) The first and last 6 countries ranked by their alignment scores with DeepSeek-R1's opinion distributions, the point represents the average alignment score of the country on all questions and the line represents the standard deviation. *** denotes $p$-value $<$ 0.001 ($t$-test). (c) The relationship of different countries to DeepSeek-R1's opinion distribution and the average human opinion distribution alignment scores. For visibility, we hide some countries. See Appendix \ref{['sec:alignment_difference_level']} for the full version.
• Figure 3: The trend of the average alignment scores of GPT-5 and DeepSeek-R1 with countries filtered using Eq. (\ref{['eq:filter']}) across waves. The shaded area indicates the standard deviation of the alignment scores at the current wave.
• Figure 4: Alignment scores among different countries. For comparison, we also show the alignment scores between the countries and DeepSeek-R1 (DS-R1).
• Figure 5: The alignment scores between Aya23 and different countries.