Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?

TL;DR

This work investigates whether LLMs’ stated values align with actions informed by those values, introducing ValueActionLens and the VIA dataset, which spans $132$ scenarios across $12$ cultures and $11$ topics with $56$ values. It defines three alignment measures—Value-Action Alignment Rate, Alignment Distance, and Alignment Ranking—and demonstrates substantial, context-dependent value-action gaps across seven models, with reasoned explanations aiding predictive modeling of these gaps. The study reveals potential real-world harms from misalignment and emphasizes the need for context-aware evaluations that go beyond traditional task performance. By releasing VIA and offering a framework for scenario-aware, pluralistic value assessment, the work provides a foundation for safer, more transparent alignment analyses in LLM deployments.

Abstract

Existing research primarily evaluates the values of LLMs by examining their stated inclinations towards specific values. However, the "Value-Action Gap," a phenomenon rooted in environmental and social psychology, reveals discrepancies between individuals' stated values and their actions in real-world contexts. To what extent do LLMs exhibit a similar gap between their stated values and their actions informed by those values? This study introduces ValueActionLens, an evaluation framework to assess the alignment between LLMs' stated values and their value-informed actions. The framework encompasses the generation of a dataset comprising 14.8k value-informed actions across twelve cultures and eleven social topics, and two tasks to evaluate how well LLMs' stated value inclinations and value-informed actions align across three different alignment measures. Extensive experiments reveal that the alignment between LLMs' stated values and actions is sub-optimal, varying significantly across scenarios and models. Analysis of misaligned results identifies potential harms from certain value-action gaps. To predict the value-action gaps, we also uncover that leveraging reasoned explanations improves performance. These findings underscore the risks of relying solely on the LLMs' stated values to predict their behaviors and emphasize the importance of context-aware evaluations of LLM values and value-action gaps.

Figures (18)

• Figure 1: An illustrative example of a "Value-Action Gap" in LLM. We observed a misalignment when prompting LLM to 1) state their inclination (i.e., Disagree) and 2) select their value-informed action (i.e., Agree), indicating 3) value-action gap towards the value of 'Social Power' in a scenario of Health in Nigeria.
• Figure 2: We introduce the ValueActionLens framework to assess the alignment between LLMs' stated values and their actions informed by those values. The framework encompasses (1) the data generation of value-informed actions across diverse cultural and social contexts; (2) two tasks for evaluating LLMs' stated values (i.e., Task1) and value-informed actions (i.e., Task2); and (3) three measures to evaluate their value-action alignment, including value-action alignment rate, alignment distance, and alignment ranking.
• Figure 3: The human-in-the-loop process of generating value-informed actions with three steps: (1) build prompt variants; (2) optimal prompt selection by AI experts; and (3) assessment of data quality by humans with diverse cultures. We show the optimal prompt and example of generated data format in Figure \ref{['fig:prompt_example']}.
• Figure 4: Heatmap of Value-Action distance across different countries and values on GPT4o-mini model.
• Figure 5: Comparing the Alignment Ranking of 56 values in Philippines (top) and United States (bottom).