Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applications

Raviraj Joshi, Rakesh Paul, Kanishk Singla, Anusha Kamath, Michael Evans, Katherine Luna, Shaona Ghosh, Utkarsh Vaidya, Eileen Long, Sanjay Singh Chauhan, Niranjan Wartikar

TL;DR

CultureGuard tackles the multilingual safety gap by proposing a scalable four-stage synthetic data pipeline that culturally adapts English safety data and translates it into multiple languages, enabling robust cross-lingual safety for LLMs. Central innovations include a cultural adaptation step using Mixtral, a cross-lingual safety consistency filter, and jail-breaking synthetic data to strengthen detection of adversarial prompts, all evaluated through LoRA-fine-tuned Llama-3.1 models. The Nemotron-Safety-Guard-Dataset-v3 (386,661 samples across 9 languages) and the Llama-3.1-Nemotron-Safety-Guard-8B-v3 model achieve state-of-the-art multilingual safety performance and strong zero-shot generalization to unseen languages, with commercial-friendly licensing to facilitate deployment. The work also reveals safety gaps in open multilingual LLMs and demonstrates the value of multilingual, culturally aware guard models for responsible global AI deployment.

Abstract

The increasing use of Large Language Models (LLMs) in agentic applications highlights the need for robust safety guard models. While content safety in English is well-studied, non-English languages lack similar advancements due to the high cost of collecting culturally aligned labeled datasets. We present CultureGuard, a novel solution for curating culturally aligned, high-quality safety datasets across multiple languages. Our approach introduces a four-stage synthetic data generation and filtering pipeline: cultural data segregation, cultural data adaptation, machine translation, and quality filtering. This pipeline enables the conversion and expansion of the Nemotron-Content-Safety-Dataset-V2 English safety dataset into eight distinct languages: Arabic, German, Spanish, French, Hindi, Japanese, Thai, and Chinese. The resulting dataset, Nemotron-Safety-Guard-Dataset-v3, comprises 386,661 samples in 9 languages and facilitates the training of Llama-3.1-Nemotron-Safety-Guard-8B-v3 via LoRA-based fine-tuning. The final model achieves state-of-the-art performance on several multilingual content safety benchmarks. Furthermore, we show our moderately multilingual fine-tuning enables robust cross-lingual transfer and strong zero-shot generalization to unseen languages. We also benchmark the latest open LLMs on multilingual safety and observe that these LLMs are more prone to give unsafe responses when prompted in non-English languages. This work advances multilingual LLM safety by enabling the development of culturally aware safety guard models.

CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applications

TL;DR

CultureGuard tackles the multilingual safety gap by proposing a scalable four-stage synthetic data pipeline that culturally adapts English safety data and translates it into multiple languages, enabling robust cross-lingual safety for LLMs. Central innovations include a cultural adaptation step using Mixtral, a cross-lingual safety consistency filter, and jail-breaking synthetic data to strengthen detection of adversarial prompts, all evaluated through LoRA-fine-tuned Llama-3.1 models. The Nemotron-Safety-Guard-Dataset-v3 (386,661 samples across 9 languages) and the Llama-3.1-Nemotron-Safety-Guard-8B-v3 model achieve state-of-the-art multilingual safety performance and strong zero-shot generalization to unseen languages, with commercial-friendly licensing to facilitate deployment. The work also reveals safety gaps in open multilingual LLMs and demonstrates the value of multilingual, culturally aware guard models for responsible global AI deployment.

Abstract

The increasing use of Large Language Models (LLMs) in agentic applications highlights the need for robust safety guard models. While content safety in English is well-studied, non-English languages lack similar advancements due to the high cost of collecting culturally aligned labeled datasets. We present CultureGuard, a novel solution for curating culturally aligned, high-quality safety datasets across multiple languages. Our approach introduces a four-stage synthetic data generation and filtering pipeline: cultural data segregation, cultural data adaptation, machine translation, and quality filtering. This pipeline enables the conversion and expansion of the Nemotron-Content-Safety-Dataset-V2 English safety dataset into eight distinct languages: Arabic, German, Spanish, French, Hindi, Japanese, Thai, and Chinese. The resulting dataset, Nemotron-Safety-Guard-Dataset-v3, comprises 386,661 samples in 9 languages and facilitates the training of Llama-3.1-Nemotron-Safety-Guard-8B-v3 via LoRA-based fine-tuning. The final model achieves state-of-the-art performance on several multilingual content safety benchmarks. Furthermore, we show our moderately multilingual fine-tuning enables robust cross-lingual transfer and strong zero-shot generalization to unseen languages. We also benchmark the latest open LLMs on multilingual safety and observe that these LLMs are more prone to give unsafe responses when prompted in non-English languages. This work advances multilingual LLM safety by enabling the development of culturally aware safety guard models.

Paper Structure

This paper contains 19 sections, 11 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Cultural Adaptation Pipeline
  5. Cultural Data Segregation
  6. Cultural Data Adaptation
  7. Machine Translation
  8. Quality Filtering
  9. Jail-Breaking Synthetic Data Curation
  10. Training and Evaluation
  11. Results and Discussion
  12. Impact of Multilingual Data
  13. Impact of Cultural Adaptation
  14. Impact of Jail-Breaking Synthetic Data and Filtering
  15. Zero-Shot Generalization to Unseen Languages
  16. ...and 4 more sections

Figures (11)

  • Figure 1: A comparison of a multilingual safety guard model's performance across different languages.
  • Figure 2: Illustration of CultureGuard's core SDG pipeline: Cultural data segregation, adaptation, and translation.
  • Figure 3: Overview of the Nemotron-Safety-Guard-Dataset-v3 (CultureGuard dataset), showing sample counts across train, test, and validation splits, derived from various sources through the CultureGuard pipeline and Jail-break Synthetic Data Generation (JB SDG).
  • Figure 4: The proposed CultureGuard pipeline for culturally aligned multilingual safety data curation.
  • Figure 5: The vanilla (non-cultural) multilingual safety data curation pipeline using translation and filtering.
  • ...and 6 more figures