Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis

Ruixiang Feng, Shen Gao, Xiuying Chen, Lisi Chen, Shuo Shang

TL;DR

CulFiT addresses cultural bias in LLMs by introducing a target-aware, multilingual training paradigm that leverages fine-grained critique data synthesis and a robust reward system. The approach decomposes cultural knowledge into verifiable units, generates multilingual critiques, and uses a two-stage SFT+DPO pipeline to align model outputs with diverse cultural norms. A new multilingual benchmark, GlobalCultureQA, alongside existing cultural benchmarks, demonstrates that CulFiT achieves state-of-the-art open-source performance in cultural alignment and general reasoning, while Hofstede-based analyses show improved cross-cultural value alignment. The work highlights the importance of multilingual data, interpretable evaluation metrics, and targeted feedback in building more inclusive and reliable AI systems, while noting computational costs and data coverage as ongoing challenges.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet they often exhibit a specific cultural biases, neglecting the values and linguistic diversity of low-resource regions. This cultural bias not only undermines universal equality, but also risks reinforcing stereotypes and perpetuating discrimination. To address this, we propose CulFiT, a novel culturally-aware training paradigm that leverages multilingual data and fine-grained reward modeling to enhance cultural sensitivity and inclusivity. Our approach synthesizes diverse cultural-related questions, constructs critique data in culturally relevant languages, and employs fine-grained rewards to decompose cultural texts into verifiable knowledge units for interpretable evaluation. We also introduce GlobalCultureQA, a multilingual open-ended question-answering dataset designed to evaluate culturally-aware responses in a global context. Extensive experiments on three existing benchmarks and our GlobalCultureQA demonstrate that CulFiT achieves state-of-the-art open-source model performance in cultural alignment and general reasoning.

CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis

TL;DR

CulFiT addresses cultural bias in LLMs by introducing a target-aware, multilingual training paradigm that leverages fine-grained critique data synthesis and a robust reward system. The approach decomposes cultural knowledge into verifiable units, generates multilingual critiques, and uses a two-stage SFT+DPO pipeline to align model outputs with diverse cultural norms. A new multilingual benchmark, GlobalCultureQA, alongside existing cultural benchmarks, demonstrates that CulFiT achieves state-of-the-art open-source performance in cultural alignment and general reasoning, while Hofstede-based analyses show improved cross-cultural value alignment. The work highlights the importance of multilingual data, interpretable evaluation metrics, and targeted feedback in building more inclusive and reliable AI systems, while noting computational costs and data coverage as ongoing challenges.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet they often exhibit a specific cultural biases, neglecting the values and linguistic diversity of low-resource regions. This cultural bias not only undermines universal equality, but also risks reinforcing stereotypes and perpetuating discrimination. To address this, we propose CulFiT, a novel culturally-aware training paradigm that leverages multilingual data and fine-grained reward modeling to enhance cultural sensitivity and inclusivity. Our approach synthesizes diverse cultural-related questions, constructs critique data in culturally relevant languages, and employs fine-grained rewards to decompose cultural texts into verifiable knowledge units for interpretable evaluation. We also introduce GlobalCultureQA, a multilingual open-ended question-answering dataset designed to evaluate culturally-aware responses in a global context. Extensive experiments on three existing benchmarks and our GlobalCultureQA demonstrate that CulFiT achieves state-of-the-art open-source model performance in cultural alignment and general reasoning.

Paper Structure

This paper contains 41 sections, 18 equations, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Cultural Bias in LLM
  4. Cultural Data Synthesis
  5. Cultural Benchmarks
  6. CulFiT
  7. Overview
  8. Target-aware Data Critique
  9. Data Synthesis
  10. Critique Generation
  11. Multi-lingual Data Synthesis
  12. Fine-grained Model Training
  13. Fine-grained Reward Modeling
  14. Cultural Precision Metric
  15. Cultural Recall Metric
  16. ...and 26 more sections

Figures (8)

  • Figure 1: An example of language inconsistency. When asked cultural-specific questions, LLMs can generate correct answers in the local language but fail to provide appropriate responses in English.
  • Figure 2: The overview of our proposed CulFiT.
  • Figure 3: Results of the precision on different reward threshold $S_{f1}$ from 0.5 to 1.0 with an interval of 0.1.
  • Figure 4: Comparison in terms of Hofstede distance.
  • Figure 5: Results on multi-lingual inconsistent rate.
  • ...and 3 more figures