Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations

Myra Cheng, Vinodkumar Prabhakaran, Alice Oh, Hayk Stepanyan, Aishwarya Verma, Charu Kalia, Erin MacMurray van Liemt, Sunipa Dev

TL;DR

This work addresses the challenge of evaluating cross-cultural norm adherence in LLMs by introducing Cultural Compass, a taxonomy that decomposes norms into contextual frame, norm specification, and mechanism of enforcement. It operationalizes a four-step norm-violation detection pipeline and demonstrates its use with the NormAd dataset across multiple industry-leading LLMs, revealing frequent norm violations that vary with country context, interaction type, and prompt intent. By enabling open-ended, context-sensitive evaluation in naturalistic conversations, the framework provides a nuanced tool for assessing cultural safety and guiding safer AI deployments. The approach emphasizes extensibility to new norms and datasets, while acknowledging limitations such as reliance on existing norm datasets and the need for human expert validation to corroborate automated judgments.

Abstract

Generative AI models ought to be useful and safe across cross-cultural contexts. One critical step toward this goal is understanding how AI models adhere to sociocultural norms. While this challenge has gained attention in NLP, existing work lacks both nuance and coverage in understanding and evaluating models' norm adherence. We address these gaps by introducing a taxonomy of norms that clarifies their contexts (e.g., distinguishing between human-human norms that models should recognize and human-AI interactional norms that apply to the human-AI interaction itself), specifications (e.g., relevant domains), and mechanisms (e.g., modes of enforcement). We demonstrate how our taxonomy can be operationalized to automatically evaluate models' norm adherence in naturalistic, open-ended settings. Our exploratory analyses suggest that state-of-the-art models frequently violate norms, though violation rates vary by model, interactional context, and country. We further show that violation rates also vary by prompt intent and situational framing. Our taxonomy and demonstrative evaluation pipeline enable nuanced, context-sensitive evaluation of cultural norm adherence in realistic settings.

Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations

TL;DR

This work addresses the challenge of evaluating cross-cultural norm adherence in LLMs by introducing Cultural Compass, a taxonomy that decomposes norms into contextual frame, norm specification, and mechanism of enforcement. It operationalizes a four-step norm-violation detection pipeline and demonstrates its use with the NormAd dataset across multiple industry-leading LLMs, revealing frequent norm violations that vary with country context, interaction type, and prompt intent. By enabling open-ended, context-sensitive evaluation in naturalistic conversations, the framework provides a nuanced tool for assessing cultural safety and guiding safer AI deployments. The approach emphasizes extensibility to new norms and datasets, while acknowledging limitations such as reliance on existing norm datasets and the need for human expert validation to corroborate automated judgments.

Abstract

Generative AI models ought to be useful and safe across cross-cultural contexts. One critical step toward this goal is understanding how AI models adhere to sociocultural norms. While this challenge has gained attention in NLP, existing work lacks both nuance and coverage in understanding and evaluating models' norm adherence. We address these gaps by introducing a taxonomy of norms that clarifies their contexts (e.g., distinguishing between human-human norms that models should recognize and human-AI interactional norms that apply to the human-AI interaction itself), specifications (e.g., relevant domains), and mechanisms (e.g., modes of enforcement). We demonstrate how our taxonomy can be operationalized to automatically evaluate models' norm adherence in naturalistic, open-ended settings. Our exploratory analyses suggest that state-of-the-art models frequently violate norms, though violation rates vary by model, interactional context, and country. We further show that violation rates also vary by prompt intent and situational framing. Our taxonomy and demonstrative evaluation pipeline enable nuanced, context-sensitive evaluation of cultural norm adherence in realistic settings.
Paper Structure (23 sections, 7 figures, 2 tables)

This paper contains 23 sections, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Taxonomy of Socio-Cultural Norms for LLM Evaluation
  4. Contextual Frame:
  5. Norm Specification:
  6. Mechanism of Enforcement:
  7. Evaluating Model Adherence to Norms
  8. Norm Violation Detection Pipeline
  9. Step 1: Generating diverse test prompts
  10. Step 2: Obtaining model responses
  11. Step 3: Surfacing relevant norms
  12. Step 4: Detecting norm violations
  13. Demonstrative Application
  14. Experimental Setup
  15. Results
  16. ...and 8 more sections

Figures (7)

  • Figure 1: Our taxonomy of norms includes contextual frame, specification, and mechanism, and grounds our evaluation pipeline for norm adherence.
  • Figure 2: Rates of violation of norms by models differ by (top) the user intent encoded into the initial prompt or query, and (bottom) the situational context of the norm as determined by our taxonomy.
  • Figure 3: Examples of free form chatbot interactions where societal norms are violated by model suggestions.
  • Figure 4: Surfacing relevant norms(top-bottom): 1. H-H norms and 2. H-AI norms.
  • Figure 5: Autorater prompt to detect model violations of surfaced norms (top-bottom): 1. H-H norms and 2. H-AI norms.
  • ...and 2 more figures