Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs generate structurally realistic social networks but overestimate political homophily

Serina Chang, Alicja Chaszczewicz, Emma Wang, Maya Josifovska, Emma Pierson, Jure Leskovec

TL;DR

This work investigates whether large language models (LLMs) can generate realistic social networks without training or explicit parameters, and whether they reproduce demographic biases like homophily. It introduces three zero-shot prompting methods—Global, Local, and Sequential—for constructing networks from fictional personas, then benchmarks the generated networks against eight real friendship networks on structural metrics (density, clustering, LCC, path length, modularity, degree distribution) and on homophily across demographic attributes. The results show that Local and Sequential prompting produce networks that closely match real networks across most structural metrics, and they reveal that political homophily dominates other demographics, with LLMs significantly overestimating cross-party ties. Incorporating LLM-generated interests does not curb political homophily, as interests encode strong political stereotypes. The findings highlight the potential of LLMs for zero-shot social-network generation while warning of bias risks and outlining directions for improving realism, diversity, and fairness in synthetic networks.

Abstract

Generating social networks is essential for many applications, such as epidemic modeling and social simulations. The emergence of generative AI, especially large language models (LLMs), offers new possibilities for social network generation: LLMs can generate networks without additional training or need to define network parameters, and users can flexibly define individuals in the network using natural language. However, this potential raises two critical questions: 1) are the social networks generated by LLMs realistic, and 2) what are risks of bias, given the importance of demographics in forming social ties? To answer these questions, we develop three prompting methods for network generation and compare the generated networks to a suite of real social networks. We find that more realistic networks are generated with "local" methods, where the LLM constructs relations for one persona at a time, compared to "global" methods that construct the entire network at once. We also find that the generated networks match real networks on many characteristics, including density, clustering, connectivity, and degree distribution. However, we find that LLMs emphasize political homophily over all other types of homophily and significantly overestimate political homophily compared to real social networks.

LLMs generate structurally realistic social networks but overestimate political homophily

TL;DR

This work investigates whether large language models (LLMs) can generate realistic social networks without training or explicit parameters, and whether they reproduce demographic biases like homophily. It introduces three zero-shot prompting methods—Global, Local, and Sequential—for constructing networks from fictional personas, then benchmarks the generated networks against eight real friendship networks on structural metrics (density, clustering, LCC, path length, modularity, degree distribution) and on homophily across demographic attributes. The results show that Local and Sequential prompting produce networks that closely match real networks across most structural metrics, and they reveal that political homophily dominates other demographics, with LLMs significantly overestimating cross-party ties. Incorporating LLM-generated interests does not curb political homophily, as interests encode strong political stereotypes. The findings highlight the potential of LLMs for zero-shot social-network generation while warning of bias risks and outlining directions for improving realism, diversity, and fairness in synthetic networks.

Abstract

Generating social networks is essential for many applications, such as epidemic modeling and social simulations. The emergence of generative AI, especially large language models (LLMs), offers new possibilities for social network generation: LLMs can generate networks without additional training or need to define network parameters, and users can flexibly define individuals in the network using natural language. However, this potential raises two critical questions: 1) are the social networks generated by LLMs realistic, and 2) what are risks of bias, given the importance of demographics in forming social ties? To answer these questions, we develop three prompting methods for network generation and compare the generated networks to a suite of real social networks. We find that more realistic networks are generated with "local" methods, where the LLM constructs relations for one persona at a time, compared to "global" methods that construct the entire network at once. We also find that the generated networks match real networks on many characteristics, including density, clustering, connectivity, and degree distribution. However, we find that LLMs emphasize political homophily over all other types of homophily and significantly overestimate political homophily compared to real social networks.
Paper Structure (60 sections, 11 equations, 22 figures, 10 tables)

This paper contains 60 sections, 11 equations, 22 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Social simulation with LLMs.
  4. LLM social biases.
  5. Graph generation.
  6. LLMs & graphs.
  7. Generating Social Networks
  8. Persona construction
  9. Network generation
  10. Global.
  11. Local.
  12. Sequential.
  13. Comparison to Real Networks
  14. Density.
  15. Average clustering coefficient.
  16. ...and 45 more sections

Figures (22)

  • Figure 1: Our three prompting methods to generate social networks with LLMs. See full prompts in Figures \ref{['fig:global-prompt']}-\ref{['fig:sequential-prompt']}.
  • Figure 2: Examples of social networks generated by our three prompting methods: Global (top), Local (middle), and Sequential (bottom).
  • Figure 3: Graph-level metrics over real and generated social networks. We visualize mean and standard error (in black) and individual data points corresponding to each network.
  • Figure 4: Degree distributions over real and generated social networks. For each set of networks, we pool degrees over nodes in the networks (Section \ref{['sec:metrics']}).
  • Figure 5: Rates of homophily in our generated networks, per demographic variable. Ratios below 1 (marked by the grey line) indicate homophily, with lower ratios indicating more homophily. We visualize mean and standard error (in black) and individual data points corresponding to each network.
  • ...and 17 more figures