Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CCD-Bench: Probing Cultural Conflict in Large Language Model Decision-Making

Hasibur Rahman, Hanan Salam

TL;DR

CCD-Bench introduces a cross-cultural value conflict benchmark to evaluate how large language models adjudicate legitimate but conflicting cultural values. The framework builds 2,182 dilemmas across seven domains, each paired with ten culturally grounded responses aligned to the GLOBE clusters, generated via a three-module pipeline and evaluated under a Stratified Latin Square design. Results show a persistent Nordic/Germanic preference in model decisions, with rationales dominated by Future Orientation and Performance Orientation and limited grounding in Assertiveness or Gender Egalitarianism, suggesting a narrow pluralism in current alignment regimes. The work highlights the need for alignment strategies that surface multiple value frames and explicitly justify trade-offs, and suggests multilingual prompts to reduce language-driven cultural bias and improve robustness in culturally sensitive decision support.

Abstract

Although large language models (LLMs) are increasingly implicated in interpersonal and societal decision-making, their ability to navigate explicit conflicts between legitimately different cultural value systems remains largely unexamined. Existing benchmarks predominantly target cultural knowledge (CulturalBench), value prediction (WorldValuesBench), or single-axis bias diagnostics (CDEval); none evaluate how LLMs adjudicate when multiple culturally grounded values directly clash. We address this gap with CCD-Bench, a benchmark that assesses LLM decision-making under cross-cultural value conflict. CCD-Bench comprises 2,182 open-ended dilemmas spanning seven domains, each paired with ten anonymized response options corresponding to the ten GLOBE cultural clusters. These dilemmas are presented using a stratified Latin square to mitigate ordering effects. We evaluate 17 non-reasoning LLMs. Models disproportionately prefer Nordic Europe (mean 20.2 percent) and Germanic Europe (12.4 percent), while options for Eastern Europe and the Middle East and North Africa are underrepresented (5.6 to 5.8 percent). Although 87.9 percent of rationales reference multiple GLOBE dimensions, this pluralism is superficial: models recombine Future Orientation and Performance Orientation, and rarely ground choices in Assertiveness or Gender Egalitarianism (both under 3 percent). Ordering effects are negligible (Cramer's V less than 0.10), and symmetrized KL divergence shows clustering by developer lineage rather than geography. These patterns suggest that current alignment pipelines promote a consensus-oriented worldview that underserves scenarios demanding power negotiation, rights-based reasoning, or gender-aware analysis. CCD-Bench shifts evaluation beyond isolated bias detection toward pluralistic decision making and highlights the need for alignment strategies that substantively engage diverse worldviews.

CCD-Bench: Probing Cultural Conflict in Large Language Model Decision-Making

TL;DR

CCD-Bench introduces a cross-cultural value conflict benchmark to evaluate how large language models adjudicate legitimate but conflicting cultural values. The framework builds 2,182 dilemmas across seven domains, each paired with ten culturally grounded responses aligned to the GLOBE clusters, generated via a three-module pipeline and evaluated under a Stratified Latin Square design. Results show a persistent Nordic/Germanic preference in model decisions, with rationales dominated by Future Orientation and Performance Orientation and limited grounding in Assertiveness or Gender Egalitarianism, suggesting a narrow pluralism in current alignment regimes. The work highlights the need for alignment strategies that surface multiple value frames and explicitly justify trade-offs, and suggests multilingual prompts to reduce language-driven cultural bias and improve robustness in culturally sensitive decision support.

Abstract

Although large language models (LLMs) are increasingly implicated in interpersonal and societal decision-making, their ability to navigate explicit conflicts between legitimately different cultural value systems remains largely unexamined. Existing benchmarks predominantly target cultural knowledge (CulturalBench), value prediction (WorldValuesBench), or single-axis bias diagnostics (CDEval); none evaluate how LLMs adjudicate when multiple culturally grounded values directly clash. We address this gap with CCD-Bench, a benchmark that assesses LLM decision-making under cross-cultural value conflict. CCD-Bench comprises 2,182 open-ended dilemmas spanning seven domains, each paired with ten anonymized response options corresponding to the ten GLOBE cultural clusters. These dilemmas are presented using a stratified Latin square to mitigate ordering effects. We evaluate 17 non-reasoning LLMs. Models disproportionately prefer Nordic Europe (mean 20.2 percent) and Germanic Europe (12.4 percent), while options for Eastern Europe and the Middle East and North Africa are underrepresented (5.6 to 5.8 percent). Although 87.9 percent of rationales reference multiple GLOBE dimensions, this pluralism is superficial: models recombine Future Orientation and Performance Orientation, and rarely ground choices in Assertiveness or Gender Egalitarianism (both under 3 percent). Ordering effects are negligible (Cramer's V less than 0.10), and symmetrized KL divergence shows clustering by developer lineage rather than geography. These patterns suggest that current alignment pipelines promote a consensus-oriented worldview that underserves scenarios demanding power negotiation, rights-based reasoning, or gender-aware analysis. CCD-Bench shifts evaluation beyond isolated bias detection toward pluralistic decision making and highlights the need for alignment strategies that substantively engage diverse worldviews.

Paper Structure

This paper contains 44 sections, 5 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Generation Module
  5. Similarity pruning and regeneration.
  6. Option ordering.
  7. Verification Module
  8. Evaluation Module
  9. Descriptive layer.
  10. Position-bias test.
  11. Domain--cluster association.
  12. Latin-square balance.
  13. Diversity metrics.
  14. Model similarity.
  15. Cluster–position bias.
  16. ...and 29 more sections

Figures (4)

  • Figure 1: Percentage distribution of cultural cluster selections across 17 LLMs. Each line represents one model, values normalized to percentages (0–100%) for each of the 10 cultural clusters.
  • Figure 2: Model-wise comparison of mention rates for nine GLOBE cultural dimensions cited in the rationales across 17 LLMs.
  • Figure 3: Selection percentage of each cultural cluster across display positions (P1–P10), aggregated over all 17 LLMs.
  • Figure 4: Heatmap of symmetrized KL-divergence between models' cultural cluster selections (left), with average-linkage hierarchical clustering dendrogram (right).