Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Monoculture in Large Language Models

Fan Wu, Emily Black, Varun Chandrasekaran

TL;DR

This work formalizes generative monoculture, a phenomenon where LLM outputs for a task lose diversity relative to the training data. It introduces a general measurement framework using source and generated distributions, attribute extraction, and dispersion metrics, and applies it to book reviews and code generation across multiple models. The study finds monoculture to be pervasive and often intensified by alignment practices like RLHF, with naive mitigation strategies offering limited relief, underscoring the need for diversity-preserving fine-tuning. The results have broad implications for education, software development, and information integrity, and the authors provide open-source tooling to support further research and mitigation efforts.

Abstract

We introduce {\em generative monoculture}, a behavior observed in large language models (LLMs) characterized by a significant narrowing of model output diversity relative to available training data for a given task: for example, generating only positive book reviews for books with a mixed reception. While in some cases, generative monoculture enhances performance (e.g., LLMs more often produce efficient code), the dangers are exacerbated in others (e.g., LLMs refuse to share diverse opinions). As LLMs are increasingly used in high-impact settings such as education and web search, careful maintenance of LLM output diversity is essential to ensure a variety of facts and perspectives are preserved over time. We experimentally demonstrate the prevalence of generative monoculture through analysis of book review and code generation tasks, and find that simple countermeasures such as altering sampling or prompting strategies are insufficient to mitigate the behavior. Moreover, our results suggest that the root causes of generative monoculture are likely embedded within the LLM's alignment processes, suggesting a need for developing fine-tuning paradigms that preserve or promote diversity.

Generative Monoculture in Large Language Models

TL;DR

This work formalizes generative monoculture, a phenomenon where LLM outputs for a task lose diversity relative to the training data. It introduces a general measurement framework using source and generated distributions, attribute extraction, and dispersion metrics, and applies it to book reviews and code generation across multiple models. The study finds monoculture to be pervasive and often intensified by alignment practices like RLHF, with naive mitigation strategies offering limited relief, underscoring the need for diversity-preserving fine-tuning. The results have broad implications for education, software development, and information integrity, and the authors provide open-source tooling to support further research and mitigation efforts.

Abstract

We introduce {\em generative monoculture}, a behavior observed in large language models (LLMs) characterized by a significant narrowing of model output diversity relative to available training data for a given task: for example, generating only positive book reviews for books with a mixed reception. While in some cases, generative monoculture enhances performance (e.g., LLMs more often produce efficient code), the dangers are exacerbated in others (e.g., LLMs refuse to share diverse opinions). As LLMs are increasingly used in high-impact settings such as education and web search, careful maintenance of LLM output diversity is essential to ensure a variety of facts and perspectives are preserved over time. We experimentally demonstrate the prevalence of generative monoculture through analysis of book review and code generation tasks, and find that simple countermeasures such as altering sampling or prompting strategies are insufficient to mitigate the behavior. Moreover, our results suggest that the root causes of generative monoculture are likely embedded within the LLM's alignment processes, suggesting a need for developing fine-tuning paradigms that preserve or promote diversity.
Paper Structure (42 sections, 29 figures, 3 tables)

This paper contains 42 sections, 29 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Defining Generative Monoculture
  3. Related Work
  4. Measuring Generative Monoculture
  5. Data Curation
  6. Attribute Extraction
  7. Metric Calculation
  8. Mitigating Generative Monoculture
  9. Experimental Setup
  10. Generating Book Reviews
  11. Generating Code Solutions
  12. Results and Takeaways
  13. Conclusion and Limitations
  14. Sampling Parameters
  15. Additional Details: Book Reviews
  16. ...and 27 more sections

Figures (29)

  • Figure 1: (Left) Comparison of the range of average-per-book sentiment scores for book reviews generated by an LLM (gen) and by human reviewers from the Goodreads dataset (src). Note the generated reviews have a much smaller range, as they are overwhelmingly positive. Model: Llama-2-chat. (Right) The spectrum of the mean pairwise Jaccard similarity among the algorithms of coding solutions. Note the generated code covers a narrower range of algorithms. Model: GPT-4
  • Figure 2: An overview of the procedure.
  • Figure 3: A summary of the scenarios, the attributes we consider, their data types, and the corresponding analysis levels as well as metrics. C and U stand for conditional and unconditional distributions.
  • Figure 4: (a-c) stacked barplots for the mean sentiment scores under varying sampling parameters, prompts, and models. For these plots, in each bar, darker hues (bottom) represent lower scores while lighter one (top) denote higher scores. See the legend for the value range of each hue. In subfigure (b), (1) and (2) refer to the two prompts as introduced in \ref{['sec:setup-goodreads']}. In subfigure (c), (a-c) refer to Llama-2-chat, Vicuna-13b, and Llama-2. (d) kernel density estimation (KDE) for the entropy values calculated on the conditional distribution of the topics. (e) unconditional topic distribution for top-10 topics. For all the subfigures, we mark the sampling parameters in them; unless marked with (1-2) or (a-c), the results are obtained on Llama-2-chat under prompt (1). These subfigures show that the model-generated reviews are overwhelmingly positive and cover a narrower range of the topics per book; moreover, there is distinctive under- and over-representation of the topics covered overall.
  • Figure 5: (Left) (a) stacked barplot for accuracy and (b) probability mass along with KDE for plagiarism scores. (Middle) Time complexity: (c) histogram of the (unconditional) distribution of the asymptotic complexity and (d) probability mass for the (conditional) distribution of entropy values. (Right) Selected code summary: (e) KDE plot for the mean pairwise cosine similarity scores for "description" as natural language and (f) stacked barplot for the mean pairwise Jaccard scores for "algorithms" as categorical values. Overall, the model-generated solutions are more accurate and efficient, display higher description similarity to each other, and cover a narrower span of algorithms. (More results in \ref{['appsec:complete-gpt4']}.)
  • ...and 24 more figures

Theorems & Definitions (1)

  • Definition 1: Generative Monoculture