Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Observing Micromotives and Macrobehavior of Large Language Models

Yuyang Cheng, Xingwei Qu, Tomas Goldsack, Chenghua Lin, Chung-Chi Chen

TL;DR

This work reframes LLM micromotives (bias and preferences) within Schelling's macrobehavior framework to study how individual, demographic-driven move decisions guided by LLMs can shape society. By replacing fixed tolerance rules with LLM-driven decisions across multiple demographic axes on a $20\times20$ grid, and evaluating via Segregation and SegShift metrics, the authors find that macrosegregation rises by ~27% regardless of the LLM used, and a tipping point emerges around $0.4$ of users following LLM advice. The study also shows that widely varying bias scores between models do not reliably predict macrobehavior, underscoring a disconnect between micromotive assessments and societal outcomes. These results highlight the need for macrobehavior-aware evaluation and caution in deploying LLMs, as increasing reliance on model guidance can drive large-scale social segregation regardless of conventional bias mitigation efforts.

Abstract

Thomas C. Schelling, awarded the 2005 Nobel Memorial Prize in Economic Sciences, pointed out that ``individuals decisions (micromotives), while often personal and localized, can lead to societal outcomes (macrobehavior) that are far more complex and different from what the individuals intended.'' The current research related to large language models' (LLMs') micromotives, such as preferences or biases, assumes that users will make more appropriate decisions once LLMs are devoid of preferences or biases. Consequently, a series of studies has focused on removing bias from LLMs. In the NLP community, while there are many discussions on LLMs' micromotives, previous studies have seldom conducted a systematic examination of how LLMs may influence society's macrobehavior. In this paper, we follow the design of Schelling's model of segregation to observe the relationship between the micromotives and macrobehavior of LLMs. Our results indicate that, regardless of the level of bias in LLMs, a highly segregated society will emerge as more people follow LLMs' suggestions. We hope our discussion will spark further consideration of the fundamental assumption regarding the mitigation of LLMs' micromotives and encourage a reevaluation of how LLMs may influence users and society.

Observing Micromotives and Macrobehavior of Large Language Models

TL;DR

This work reframes LLM micromotives (bias and preferences) within Schelling's macrobehavior framework to study how individual, demographic-driven move decisions guided by LLMs can shape society. By replacing fixed tolerance rules with LLM-driven decisions across multiple demographic axes on a grid, and evaluating via Segregation and SegShift metrics, the authors find that macrosegregation rises by ~27% regardless of the LLM used, and a tipping point emerges around of users following LLM advice. The study also shows that widely varying bias scores between models do not reliably predict macrobehavior, underscoring a disconnect between micromotive assessments and societal outcomes. These results highlight the need for macrobehavior-aware evaluation and caution in deploying LLMs, as increasing reliance on model guidance can drive large-scale social segregation regardless of conventional bias mitigation efforts.

Abstract

Thomas C. Schelling, awarded the 2005 Nobel Memorial Prize in Economic Sciences, pointed out that ``individuals decisions (micromotives), while often personal and localized, can lead to societal outcomes (macrobehavior) that are far more complex and different from what the individuals intended.'' The current research related to large language models' (LLMs') micromotives, such as preferences or biases, assumes that users will make more appropriate decisions once LLMs are devoid of preferences or biases. Consequently, a series of studies has focused on removing bias from LLMs. In the NLP community, while there are many discussions on LLMs' micromotives, previous studies have seldom conducted a systematic examination of how LLMs may influence society's macrobehavior. In this paper, we follow the design of Schelling's model of segregation to observe the relationship between the micromotives and macrobehavior of LLMs. Our results indicate that, regardless of the level of bias in LLMs, a highly segregated society will emerge as more people follow LLMs' suggestions. We hope our discussion will spark further consideration of the fundamental assumption regarding the mitigation of LLMs' micromotives and encourage a reevaluation of how LLMs may influence users and society.

Paper Structure

This paper contains 9 sections, 1 equation, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Experimental Design
  4. Question and Response Formulation
  5. Evaluation
  6. Setup
  7. Experimental Results
  8. Reflection
  9. Prompt

Figures (3)

  • Figure 1: As the number of LLM users increases, society becomes more segregated.
  • Figure 2: Comparative analysis of recent approaches for observing micromotives and Schelling's macrobehavior observation method.
  • Figure 3: The first distribution represents the probability distribution of agents moving in the Schelling model, where agents move if the probability is less than the threshold and do not move if it is greater. The setting is a 20 $\times$ 20 grid with 180 red and 180 blue agents. With a probability of 0.375, which is slightly above the theoretical threshold of 0.3, the process ends in 16 iterations, with an average similarity of 0.77.