Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

To Generate or Discriminate? Methodological Considerations for Measuring Cultural Alignment in LLMs

Saurabh Kumar Pandey, Sougata Saha, Monojit Choudhury

TL;DR

The study tackles the confounding factors in measuring LLM cultural alignment using SDP by introducing inverse SDP (ISDP), reframing the task as discrimination of demographic proxies from user behaviors. It evaluates ISDP on the Goodreads-CSI dataset with four LLMs, comparing actual versus simulated behaviors across three aggregation levels. Key findings reveal that models often distinguish actual user behavior better than simulated data, but personalization gains are limited at the individual level, and some models nonetheless reveal stereotypes in simulated settings. The work argues that ISDP complements SDP, offering a more reliable lens on cultural awareness and guiding the design of evaluation benchmarks for culturally sensitive AI.

Abstract

Socio-demographic prompting (SDP) - prompting Large Language Models (LLMs) using demographic proxies to generate culturally aligned outputs - often shows LLM responses as stereotypical and biased. While effective in assessing LLMs' cultural competency, SDP is prone to confounding factors such as prompt sensitivity, decoding parameters, and the inherent difficulty of generation over discrimination tasks due to larger output spaces. These factors complicate interpretation, making it difficult to determine if the poor performance is due to bias or the task design. To address this, we use inverse socio-demographic prompting (ISDP), where we prompt LLMs to discriminate and predict the demographic proxy from actual and simulated user behavior from different users. We use the Goodreads-CSI dataset (Saha et al., 2025), which captures difficulty in understanding English book reviews for users from India, Mexico, and the USA, and test four LLMs: Aya-23, Gemma-2, GPT-4o, and LLaMA-3.1 with ISDP. Results show that models perform better with actual behaviors than simulated ones, contrary to what SDP suggests. However, performance with both behavior types diminishes and becomes nearly equal at the individual level, indicating limits to personalization.

To Generate or Discriminate? Methodological Considerations for Measuring Cultural Alignment in LLMs

TL;DR

The study tackles the confounding factors in measuring LLM cultural alignment using SDP by introducing inverse SDP (ISDP), reframing the task as discrimination of demographic proxies from user behaviors. It evaluates ISDP on the Goodreads-CSI dataset with four LLMs, comparing actual versus simulated behaviors across three aggregation levels. Key findings reveal that models often distinguish actual user behavior better than simulated data, but personalization gains are limited at the individual level, and some models nonetheless reveal stereotypes in simulated settings. The work argues that ISDP complements SDP, offering a more reliable lens on cultural awareness and guiding the design of evaluation benchmarks for culturally sensitive AI.

Abstract

Socio-demographic prompting (SDP) - prompting Large Language Models (LLMs) using demographic proxies to generate culturally aligned outputs - often shows LLM responses as stereotypical and biased. While effective in assessing LLMs' cultural competency, SDP is prone to confounding factors such as prompt sensitivity, decoding parameters, and the inherent difficulty of generation over discrimination tasks due to larger output spaces. These factors complicate interpretation, making it difficult to determine if the poor performance is due to bias or the task design. To address this, we use inverse socio-demographic prompting (ISDP), where we prompt LLMs to discriminate and predict the demographic proxy from actual and simulated user behavior from different users. We use the Goodreads-CSI dataset (Saha et al., 2025), which captures difficulty in understanding English book reviews for users from India, Mexico, and the USA, and test four LLMs: Aya-23, Gemma-2, GPT-4o, and LLaMA-3.1 with ISDP. Results show that models perform better with actual behaviors than simulated ones, contrary to what SDP suggests. However, performance with both behavior types diminishes and becomes nearly equal at the individual level, indicating limits to personalization.
Paper Structure (13 sections, 5 figures, 3 tables)

This paper contains 13 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dataset
  4. Method
  5. Results and Analysis
  6. Discussion and Conclusion
  7. Appendix
  8. Prompts - SDP
  9. Prompts - ISDP
  10. Results: Statistical Significance
  11. Related Work
  12. Dataset standardization
  13. Span standardization:

Figures (5)

  • Figure 1: IAA between models and humans for SDP.
  • Figure 2: Experimental setup for all three levels of behavior aggregation. Phase 1: SDP and Phase 2: ISDP.
  • Figure 3: Model-wise MRR scores across five different sources of behavior and three levels of behavior aggregation. The x-axis represents the discriminators, whereas the legend represents the generators used in the experiments.
  • Figure 4: Average MRR scores for different combinations of generators and discriminators.
  • Figure 5: Average MRR scores for different levels of behavior.