Navigating the Cultural Kaleidoscope: A Hitchhiker's Guide to Sensitivity in Large Language Models

TL;DR

This work addresses cultural harm in LLM outputs, with a focus on small-parameter models lacking global cultural coverage. It introduces two resources—a cultural harm evaluation dataset and a culturally aligned preference dataset—and demonstrates that culturally informed feedback, particularly via ORPO alignment, substantially reduces harmful, culturally insensitive content across multiple languages and cultures. Through extensive single- and multi-turn evaluations across a spectrum of models, the study reveals both the variability of cultural safety by architecture and the effectiveness of targeted alignment in mitigating harm without sacrificing utility. The findings advance practical pathways for deploying inclusive, culturally respectful AI in global contexts, highlighting ORPO as a robust mechanism for culturally aligned safety.

Abstract

As LLMs are increasingly deployed in global applications, the importance of cultural sensitivity becomes paramount, ensuring that users from diverse backgrounds feel respected and understood. Cultural harm can arise when these models fail to align with specific cultural norms, resulting in misrepresentations or violations of cultural values. This work addresses the challenges of ensuring cultural sensitivity in LLMs, especially in small-parameter models that often lack the extensive training data needed to capture global cultural nuances. We present two key contributions: (1) A cultural harm test dataset, created to assess model outputs across different cultural contexts through scenarios that expose potential cultural insensitivities, and (2) A culturally aligned preference dataset, aimed at restoring cultural sensitivity through fine-tuning based on feedback from diverse annotators. These datasets facilitate the evaluation and enhancement of LLMs, ensuring their ethical and safe deployment across different cultural landscapes. Our results show that integrating culturally aligned feedback leads to a marked improvement in model behavior, significantly reducing the likelihood of generating culturally insensitive or harmful content. Ultimately, this work paves the way for more inclusive and respectful AI systems, fostering a future where LLMs can safely and ethically navigate the complexities of diverse cultural landscapes.

Figures (25)

• Figure 1: Our evaluation highlighted the disparity in cultural harms produced by Llama-2(7B) model across the globe. Shade darkness represents propensity towards cultural harm.
• Figure 2: Pie charts show the 12 main cultures in our dataset, while the list on the right outlines the key areas that could lead to potential harm within each culture.
• Figure 3: Global Set examples.
• Figure 4: Local Set examples.
• Figure 5: The bar colors represent Llama's ASR improvements for global single turn dataset: Red for vanilla, Blue for DPO, and Green for ORPO. Both plots show the ASR across multiple categories.