Cultural Learning-Based Culture Adaptation of Language Models

TL;DR

This work introduces CLCA, a culture-learning-based framework that aligns LLMs with diverse cultural values by simulating culture-adapted social interactions. It combines imitative (dialogue) and instructed (intent) learning to form a multi-task training objective and evaluates alignment using World Values Survey data across five cultures. Results show that CLCA improves culture-level alignment and individual-level accuracy across multiple model families, with social interaction data and intent understanding playing complementary roles. Multilingual transfer indicates the approach generalizes beyond English, though teacher-model quality influences data generation. The study highlights the potential of cultural learning to create more globally inclusive NLP systems while acknowledging biases in synthetic data and the need for real-world human data for further validation.

Abstract

Adapting large language models (LLMs) to diverse cultural values is a challenging task, as existing LLMs often reflect the values of specific groups by default, and potentially causing harm to others. In this paper, we present CLCA, a novel framework for enhancing LLM alignment with cultural values based on cultural learning. The framework leverages simulated social interactions to generate conversations in which LLMs engage in role-playing within culturally adapted social scenarios, capturing implicit cultural norms for model fine-tuning. CLCA improves cultural value alignment across various model architectures measured using World Value Survey data, demonstrating the effectiveness of our proposed approach. Our results provide early evidence that understanding intent and social interactions can enhance cultural value adaptation in LLMs, highlighting the promise of training approaches based on cultural learning.

Figures (6)

• Figure 1: We use culture-adapted role-playing to generate synthetic social interaction conversations. Then, the proposed cultural learning-based framework jointly trains on conversations, intents and their relevance to culture, to improve cultural value alignment.
• Figure 2: (1) The framework first automatically generates conversations through culture-adapted role-playing in social settings. (2) These conversations are then filtered using GPT models to ensure quality and relevance. (3) The filtered data is labelled with free-text intents. (4) Both the conversation and intent data are integrated into a cultural learning-based training process (CLCA). (5) The resulting models are evaluated using the World Values Survey.
• Figure 3: The individual-level accuracy (the higher the better) of CLCA versus zero-shot results of the persona baseline (Standard, described in §\ref{['subsec:method']}) against the ground truth answers from the survey for different cultures. Mistral results in Figure \ref{['fig:mistral']}, and averages for all models in Table \ref{['tab:acc_main']} in the Appendix. All models are instruction-tuned.
• Figure 4: Average performance of models (Standard is the zero-shot evaluation of the persona baseline described in §\ref{['subsec:method']}., CLCA is the adaptation in English) responding to survey questions in the native language of the culture. Results are averaged over all languages.
• Figure 5: Individual-level accuracy for Mistral model.