Benchmarking Multi-National Value Alignment for Large Language Models

TL;DR

NaVAB introduces the first cross-national benchmark for evaluating and improving LLM value alignment across five major nations by building a data-driven value assessment pipeline from cross-national news. It combines topic modeling, value-sensitive topic screening, and a graph-based Conflict Reduction strategy to generate evaluation data in the form of <Q, S, RS> triples and to support two evaluation modes: quoted statements and official media positions. Extensive experiments across diverse model families show that model type, size, and training regime influence alignment, with MoE and larger/instruction-tuned models performing better, and DPO fine-tuning further enhancing alignment when paired with Conflict Reduction. The work provides a practical, scalable framework for assessing and guiding cross-national value alignment, illuminating how language, media sources, and processing strategies impact real-world deployment of LLMs in multinational contexts.

Abstract

Do Large Language Models (LLMs) hold positions that conflict with your country's values? Occasionally they do! However, existing works primarily focus on ethical reviews, failing to capture the diversity of national values, which encompass broader policy, legal, and moral considerations. Furthermore, current benchmarks that rely on spectrum tests using manually designed questionnaires are not easily scalable. To address these limitations, we introduce NaVAB, a comprehensive benchmark to evaluate the alignment of LLMs with the values of five major nations: China, the United States, the United Kingdom, France, and Germany. NaVAB implements a national value extraction pipeline to efficiently construct value assessment datasets. Specifically, we propose a modeling procedure with instruction tagging to process raw data sources, a screening process to filter value-related topics and a generation process with a Conflict Reduction mechanism to filter non-conflicting values.We conduct extensive experiments on various LLMs across countries, and the results provide insights into assisting in the identification of misaligned scenarios. Moreover, we demonstrate that NaVAB can be combined with alignment techniques to effectively reduce value concerns by aligning LLMs' values with the target country.

Figures (7)

• Figure 1: A demonstration of differnet LLM's responses compared with people's attitude cross nations
• Figure 2: The pipeline of NaVAB. Each process is introduced in Section \ref{['pipeline']}. The final output of the value data is a triple consisting of three components: Q (Question), S(Statement), RS(Reverse Statement), which is illustrated in Section \ref{['vad']}. All processes are described step by step in Section \ref{['pipeline']}.
• Figure 3: Two examples showing the clusters from different news data sources and the top 5 topics of the corresponding clusters. Grey points are outliers explained in Section \ref{['topic_modeling']}.
• Figure 4: A comparison between traditional evaluation method and ours. MC and AJ denote Multiple-Choice and Answer Judgment, respectively. These two methods are introduced in Section \ref{['evaluation_metric']}.
• Figure 5: A case study comparing the LLM's alignment before and after fine-tuning with DPO using NaVAB's data. We use Llama3.1-8b-Instruct as the model.