Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debate

TL;DR

Western cultural bias in large language models persists across languages; the paper introduces CEBiasBench and MAV to enable explicit neutral judgments and evaluates debiasing with a training-free MACD framework. MACD uses explicit cultural personas and a SCGRD-driven deliberation to synthesize balanced responses, significantly improving No-Bias rates on CEBiasBench and generalizing to the CAMeL Arabic benchmark. The findings demonstrate that explicit cultural representation is essential for cross-cultural fairness and that language prompting alone is insufficient to erase bias. The work also highlights evaluator biases in automated judgments and offers a scalable, multilingual approach to debiasing with practical implications for inclusive AI systems.

Abstract

Large language models (LLMs) exhibit systematic Western-centric bias, yet whether prompting in non-Western languages (e.g., Chinese) can mitigate this remains understudied. Answering this question requires rigorous evaluation and effective mitigation, but existing approaches fall short on both fronts: evaluation methods force outputs into predefined cultural categories without a neutral option, while mitigation relies on expensive multi-cultural corpora or agent frameworks that use functional roles (e.g., Planner--Critique) lacking explicit cultural representation. To address these gaps, we introduce CEBiasBench, a Chinese--English bilingual benchmark, and Multi-Agent Vote (MAV), which enables explicit ``no bias'' judgments. Using this framework, we find that Chinese prompting merely shifts bias toward East Asian perspectives rather than eliminating it. To mitigate such persistent bias, we propose Multi-Agent Cultural Debate (MACD), a training-free framework that assigns agents distinct cultural personas and orchestrates deliberation via a "Seeking Common Ground while Reserving Differences" strategy. Experiments demonstrate that MACD achieves 57.6% average No Bias Rate evaluated by LLM-as-judge and 86.0% evaluated by MAV (vs. 47.6% and 69.0% baseline using GPT-4o as backbone) on CEBiasBench and generalizes to the Arabic CAMeL benchmark, confirming that explicit cultural representation in agent frameworks is essential for cross-cultural fairness.

Figures (6)

• Figure 1: (a) CEBiasBench composition: five everyday cultural domains with equal representation. (b) Example LLM responses on CEBiasBench, showing biased (red) vs. unbiased (green) outputs.
• Figure 2: Our proposed MACD and MAV framework.
• Figure 3: A bilingual example showing MACD effectiveness using GPT-4o as backbone. While GPT-4o and DeepSeek-R1 output culturally biased responses like "hotpot" and "roast chicken", MACD enables cultural agents to produce unbiased output through debate and summary. Notably, underrepresented samples like "mansaf" and "ugali" emerge naturally in MACD.
• Figure 4: Bias distribution on CAMeL benchmark (GPT-4o backbone).
• Figure 5: Bias heatmap to show the distribution of culturally biased answer.