Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting

Sagnik Mukherjee, Muhammad Farid Adilazuarda, Sunayana Sitaram, Kalika Bali, Alham Fikri Aji, Monojit Choudhury

TL;DR

The paper investigates whether socio-demographic prompting can reliably elicit cultural biases in LLMs or whether observed effects are confounded by placebo-like variations. It systematically probes four models across nine proxies (four cultural, five non-cultural) and four benchmarks (EtiCor, CALI, MMLU, ETHICS) using 45 prompt variants, yielding about 270k inferences per model. The authors compute a cross-cue sensitivity metric from response matrices and find substantial variation for nearly all models except GPT-4, challenging the robustness of culture-driven probing and highlighting placebo-like effects. They argue for stronger control designs and standardization in prompting-based bias assessment, and they release datasets and responses to facilitate replication and further research.

Abstract

Socio-demographic prompting is a commonly employed approach to study cultural biases in LLMs as well as for aligning models to certain cultures. In this paper, we systematically probe four LLMs (Llama 3, Mistral v0.2, GPT-3.5 Turbo and GPT-4) with prompts that are conditioned on culturally sensitive and non-sensitive cues, on datasets that are supposed to be culturally sensitive (EtiCor and CALI) or neutral (MMLU and ETHICS). We observe that all models except GPT-4 show significant variations in their responses on both kinds of datasets for both kinds of prompts, casting doubt on the robustness of the culturally-conditioned prompting as a method for eliciting cultural bias in models or as an alignment strategy. The work also calls rethinking the control experiment design to tease apart the cultural conditioning of responses from "placebo effect", i.e., random perturbations of model responses due to arbitrary tokens in the prompt.

Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting

TL;DR

The paper investigates whether socio-demographic prompting can reliably elicit cultural biases in LLMs or whether observed effects are confounded by placebo-like variations. It systematically probes four models across nine proxies (four cultural, five non-cultural) and four benchmarks (EtiCor, CALI, MMLU, ETHICS) using 45 prompt variants, yielding about 270k inferences per model. The authors compute a cross-cue sensitivity metric from response matrices and find substantial variation for nearly all models except GPT-4, challenging the robustness of culture-driven probing and highlighting placebo-like effects. They argue for stronger control designs and standardization in prompting-based bias assessment, and they release datasets and responses to facilitate replication and further research.

Abstract

Socio-demographic prompting is a commonly employed approach to study cultural biases in LLMs as well as for aligning models to certain cultures. In this paper, we systematically probe four LLMs (Llama 3, Mistral v0.2, GPT-3.5 Turbo and GPT-4) with prompts that are conditioned on culturally sensitive and non-sensitive cues, on datasets that are supposed to be culturally sensitive (EtiCor and CALI) or neutral (MMLU and ETHICS). We observe that all models except GPT-4 show significant variations in their responses on both kinds of datasets for both kinds of prompts, casting doubt on the robustness of the culturally-conditioned prompting as a method for eliciting cultural bias in models or as an alignment strategy. The work also calls rethinking the control experiment design to tease apart the cultural conditioning of responses from "placebo effect", i.e., random perturbations of model responses due to arbitrary tokens in the prompt.
Paper Structure (27 sections, 3 equations, 2 figures, 27 tables)

This paper contains 27 sections, 3 equations, 2 figures, 27 tables.

Table of Contents

  1. Introduction
  2. Notations, Definitions and Research Questions
  3. Experiments
  4. Proxies, Cues and Prompts
  5. Models and Datasets
  6. Inference Pipeline
  7. Computing Sensitivity
  8. Results
  9. Response Variation Across Cultural Proxies
  10. Response Variation Across All Proxies
  11. Pattern of Variance Across Models
  12. Qualitative Analysis
  13. Cultural Sensitivity of Datasets
  14. The Curious Case of Non-cultural Proxies
  15. Cultural Markers in Llama vs Mistral
  16. ...and 12 more sections

Figures (2)

  • Figure 1: Composition of lexical variation, proxy, model instruction and datapoint to get the final model input
  • Figure 2: Heatmaps showing Pearson correlation coefficients for four different models. The upper triangle in each heatmap represents the Ethics dataset, while the lower triangle represents the MMLU dataset. The models included are Llama3-8B, Mistral 7B, GPT-3.5, and GPT-4.