Self-Pluralising Culture Alignment for Large Language Models

TL;DR

CultureSPA is proposed, a Self-Pluralising Culture Alignment framework that allows LLMs to simultaneously align to pluralistic cultures and significantly improves the alignment of LLMs to diverse cultures without compromising general abilities.

Abstract

As large language models (LLMs) become increasingly accessible in many countries, it is essential to align them to serve pluralistic human values across cultures. However, pluralistic culture alignment in LLMs remain an open problem. In this paper, we propose CultureSPA, a Self-Pluralising Culture Alignment framework that allows LLMs to simultaneously align to pluralistic cultures. The framework first generates questions on various culture topics, then yields LLM outputs in response to these generated questions under both culture-aware and culture-unaware settings. By comparing culture-aware/unaware outputs, we are able to detect and collect culture-related instances. These instances are employed to fine-tune LLMs to serve pluralistic cultures in either a culture-joint or culture-specific way. Extensive experiments demonstrate that CultureSPA significantly improves the alignment of LLMs to diverse cultures without compromising general abilities. And further improvements can be achieved if CultureSPA is combined with advanced prompt engineering techniques. Comparisons between culture-joint and culture-specific tuning strategies, along with variations in data quality and quantity, illustrate the robustness of our method. We also explore the mechanisms underlying CultureSPA and the relations between different cultures it reflects.

Figures (6)

• Figure 1: Cultural alignment scores of LLaMA3 across various countries. Culture-Unaware/Aware Prompting: The model isn't/is prompted to align with the target culture. CultureSPA: The model is fine-tuned with the proposed self-pluralising culture alignment. Country names are standardized according to the ISO 3166-1 alpha-3 country codes.
• Figure 2: Diagram of the proposed CultureSPA. The framework consists of 4 key steps. In the first step, it generates diverse culture-related questions on 13 culture topics from 260 seed questions collected from WVS. It then collects LLM outputs for these questions under two scenarios: culture-unaware prompting and culture-aware prompting. Samples that demonstrate output shifts between the two scenarios are considered the most representative of the corresponding culture and hence collected in Step 3. Finally, the collected culture-related QA pairs (Question+CAP output) are employed for culture-joint/specific SFT.
• Figure 3: Distribution of topics and cultures in the activation data generated by LLaMA-3-8B-Instruct.
• Figure 4: Comparison of different data sampling strategies. With the P1 baseline as a reference, changes in cultural alignment scores achieved by each strategy are reported. "CRQPC" refers to our proposed Culture-Related QA Pairs Collecting, "RDS" refers to Random Data Sampling, and "CDS" refers to Consistent Data Sampling, which is the opposite of CRQPC.
• Figure 5: Cross-cultural alignment scores for the WVS reference and LLM outputs across three methods, along with their correlation coefficients with the reference distribution.