Rethinking LLM Human Simulation: When a Graph is What You Need

TL;DR

This work challenges the necessity of large language models for discrete-choice human simulation by introducing Graph-basEd Models for human Simulation (GEMS), a heterogeneous-graph approach that casts prediction of user choices as link prediction. The GNN-based framework learns from relational structure among subgroups, individuals, and choices, using language representations only when needed (Setting 3). Across three datasets and three task settings, GEMS matches or surpasses strong LLM baselines while delivering substantial gains in efficiency, interpretability, and transparency, due to a much smaller parameter footprint and explicit relational inductive biases. The results suggest graph-based modeling as a practical, scalable alternative to LLMs for a broad class of human-simulation problems involving discrete choices, with code available at the authors' repository.

Abstract

Large language models (LLMs) are increasingly used to simulate humans, with applications ranging from survey prediction to decision-making. However, are LLMs strictly necessary, or can smaller, domain-grounded models suffice? We identify a large class of simulation problems in which individuals make choices among discrete options, where a graph neural network (GNN) can match or surpass strong LLM baselines despite being three orders of magnitude smaller. We introduce Graph-basEd Models for human Simulation (GEMS), which casts discrete choice simulation tasks as a link prediction problem on graphs, leveraging relational knowledge while incorporating language representations only when needed. Evaluations across three key settings on three simulation datasets show that GEMS achieves comparable or better accuracy than LLMs, with far greater efficiency, interpretability, and transparency, highlighting the promise of graph-based modeling as a lightweight alternative to LLMs for human simulation. Our code is available at https://github.com/schang-lab/gems.

Figures (6)

• Figure 1: In our GEMS framework, we construct a heterogeneous graph for discrete choice simulation tasks (Top) where the goal is to predict the option chosen by an individual in response to a context or question. Under three widely studied settings (Bottom), we show that our GNN-based method achieves prediction accuracy consistently comparable to the best LLM-based approaches.
• Figure 2: Overall architecture of GEMS. The graph encoder learns representations of individual and choice nodes from the relational structure of observed responses, then predicts new responses with a softmax classifier over question options (Top). In setting 3 only, we learn a simple LLM-to-GNN projection that maps an LLM's frozen representation of the choice node's text to its GNN embedding, so that we can acquire meaningful representations of new questions (Bottom).
• Figure 3: Prediction accuracy vs. GPU-hours (A100-80GB-SXM4) on the OpinionQA dataset by task setting and method. Zero-/few-shot prompting accuracies fall below the plotted y-range. For LLM-based methods, we report the best result across three LLMs (LLaMA-2-7B, Mistral-7B-v0.1, and Qwen3-8B). For GEMS, we report the best result across three models (RGCN, GAT, and SAGE) for setting 1 & 2, and report across different LLMs for setting 3. See \ref{['appendix:training_details']} for details.
• Figure 4: Visualization of LLM hidden states and GNN node embeddings on the first and second components of principal component analysis.
• Figure 5: Mean and standard deviation of prediction accuracy on setting 3 (new questions) of OpinionQA dataset when extracting hidden states from different layers of Mistral-7B-v0.1 (\ref{['table:table_inductive_question']}). Layer 0 is the post-embedding activation and layer 32 is the final pre-LM head activation.