Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Attributing Culture-Conditioned Generations to Pretraining Corpora

Huihan Li, Arnav Goel, Keyu He, Xiang Ren

TL;DR

This paper tackles culture-conditioned generation biases by introducing MEMOed, a framework that attributes model outputs to memorization from pretraining data. By analyzing 110 cultures across food and clothing with the OLMo-7B model and Dolma pretraining data, MEMOed identifies four symbol-culture association types and computes a Contribution Score to detect memorization via a z-score outlier approach. The findings show memorized associations are richer for well-represented cultures, while diffuse associations driven by high-frequency symbols dominate outputs, and cross-cultural generalization occurs more when cultures are frequent in pretraining. Weak associations generalize in ways not tightly tied to memorization, underscoring biases in culture recall and the need for targeted data or unlearning to improve cross-cultural fidelity in generation.

Abstract

In open-ended generative tasks like narrative writing or dialogue, large language models often exhibit cultural biases, showing limited knowledge and generating templated outputs for less prevalent cultures. Recent works show that these biases may stem from uneven cultural representation in pretraining corpora. This work investigates how pretraining leads to biased culture-conditioned generations by analyzing how models associate entities with cultures based on pretraining data patterns. We propose the MEMOed framework (MEMOrization from pretraining document) to determine whether a generation for a culture arises from memorization. Using MEMOed on culture-conditioned generations about food and clothing for 110 cultures, we find that high-frequency cultures in pretraining data yield more generations with memorized symbols, while some low-frequency cultures produce none. Additionally, the model favors generating entities with extraordinarily high frequency regardless of the conditioned culture, reflecting biases toward frequent pretraining terms irrespective of relevance. We hope that the MEMOed framework and our insights will inspire more works on attributing model performance on pretraining data.

Attributing Culture-Conditioned Generations to Pretraining Corpora

TL;DR

This paper tackles culture-conditioned generation biases by introducing MEMOed, a framework that attributes model outputs to memorization from pretraining data. By analyzing 110 cultures across food and clothing with the OLMo-7B model and Dolma pretraining data, MEMOed identifies four symbol-culture association types and computes a Contribution Score to detect memorization via a z-score outlier approach. The findings show memorized associations are richer for well-represented cultures, while diffuse associations driven by high-frequency symbols dominate outputs, and cross-cultural generalization occurs more when cultures are frequent in pretraining. Weak associations generalize in ways not tightly tied to memorization, underscoring biases in culture recall and the need for targeted data or unlearning to improve cross-cultural fidelity in generation.

Abstract

In open-ended generative tasks like narrative writing or dialogue, large language models often exhibit cultural biases, showing limited knowledge and generating templated outputs for less prevalent cultures. Recent works show that these biases may stem from uneven cultural representation in pretraining corpora. This work investigates how pretraining leads to biased culture-conditioned generations by analyzing how models associate entities with cultures based on pretraining data patterns. We propose the MEMOed framework (MEMOrization from pretraining document) to determine whether a generation for a culture arises from memorization. Using MEMOed on culture-conditioned generations about food and clothing for 110 cultures, we find that high-frequency cultures in pretraining data yield more generations with memorized symbols, while some low-frequency cultures produce none. Additionally, the model favors generating entities with extraordinarily high frequency regardless of the conditioned culture, reflecting biases toward frequent pretraining terms irrespective of relevance. We hope that the MEMOed framework and our insights will inspire more works on attributing model performance on pretraining data.
Paper Structure (50 sections, 1 equation, 17 figures, 11 tables, 1 algorithm)

This paper contains 50 sections, 1 equation, 17 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Memorization and Generalization.
  4. Culture bias in culture-conditioned generation tasks.
  5. Analysis Setup
  6. Types of Symbol-Culture Associations in Culture-conditioned Generations
  7. Data Collection Process
  8. Model and Data.
  9. Scope.
  10. Generation.
  11. Identifying Knowledge Memorization from Culture-conditioned Generations
  12. Contribution Score.
  13. Determining memorization with z-score.
  14. Analysis on Non-Memorized Associations from Culture-Conditioned Generations
  15. Diffuse Association.
  16. ...and 35 more sections

Figures (17)

  • Figure 1: Four types of culture-symbol associations in culture-conditioned generations
  • Figure 2: MEMOed pipeline, demonstrated with Malaysian culture on food topic.
  • Figure 3: Higher contribution score means stronger evidence of culture/symbol association in pretraining data, as defined in §\ref{['subsec:memorization']}. Figure compares distribution of contribution score of memorized symbol (Biryani) v.s. non-memorized symbol (Roast turkey). Y-axis shows all cultures for which the symbol is generated. Red font show the z-score: $\geq$ 2.6 means memorization.
  • Figure 4: Geographical Distribution of Memorized Association
  • Figure 5: Overshadowing ratio $r$ of all diffuse association for topic clothing.
  • ...and 12 more figures