Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice

Jian-Qiao Zhu, Haijiang Yan, Thomas L. Griffiths

TL;DR

This work tests whether a small language model pretrained on ecologically valid arithmetic can serve as a cognitive model of human risky and intertemporal choices. By focusing on the computationally equivalent tasks of $EV(A)=\sum_i p_i x_i$ and $PV(A)=\sum_t d^{t} x_t$, the authors train Arithmetic-GPT (10M parameters) on synthetic EV/PV data and evaluate its embeddings against human data. The results show that ecologically distributed pretraining yields embeddings that predict human choices better than many classical cognitive models and even some open-weight LLMs, with robust evidence from cross-validation and ablation analyses. These findings highlight the crucial role of training data distributions in shaping cognitive alignment and suggest a programmable path to using LLMs as cognitive probes, while calling for thorough data-disclosure and further exploration of alternative architectures and tasks.

Abstract

The observed similarities in the behavior of humans and Large Language Models (LLMs) have prompted researchers to consider the potential of using LLMs as models of human cognition. However, several significant challenges must be addressed before LLMs can be legitimately regarded as cognitive models. For instance, LLMs are trained on far more data than humans typically encounter, and may have been directly trained on human data in specific cognitive tasks or aligned with human preferences. Consequently, the origins of these behavioral similarities are not well understood. In this paper, we propose a novel way to enhance the utility of LLMs as cognitive models. This approach involves (i) leveraging computationally equivalent tasks that both an LLM and a rational agent need to master for solving a cognitive problem and (ii) examining the specific task distributions required for an LLM to exhibit human-like behaviors. We apply this approach to decision-making -- specifically risky and intertemporal choice -- where the key computationally equivalent task is the arithmetic of expected value calculations. We show that an LLM pretrained on an ecologically valid arithmetic dataset, which we call Arithmetic-GPT, predicts human behavior better than many traditional cognitive models. Pretraining LLMs on ecologically valid arithmetic datasets is sufficient to produce a strong correspondence between these models and human decision-making. Our results also suggest that LLMs used as cognitive models should be carefully investigated via ablation studies of the pretraining data.

Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice

TL;DR

This work tests whether a small language model pretrained on ecologically valid arithmetic can serve as a cognitive model of human risky and intertemporal choices. By focusing on the computationally equivalent tasks of and , the authors train Arithmetic-GPT (10M parameters) on synthetic EV/PV data and evaluate its embeddings against human data. The results show that ecologically distributed pretraining yields embeddings that predict human choices better than many classical cognitive models and even some open-weight LLMs, with robust evidence from cross-validation and ablation analyses. These findings highlight the crucial role of training data distributions in shaping cognitive alignment and suggest a programmable path to using LLMs as cognitive probes, while calling for thorough data-disclosure and further exploration of alternative architectures and tasks.

Abstract

The observed similarities in the behavior of humans and Large Language Models (LLMs) have prompted researchers to consider the potential of using LLMs as models of human cognition. However, several significant challenges must be addressed before LLMs can be legitimately regarded as cognitive models. For instance, LLMs are trained on far more data than humans typically encounter, and may have been directly trained on human data in specific cognitive tasks or aligned with human preferences. Consequently, the origins of these behavioral similarities are not well understood. In this paper, we propose a novel way to enhance the utility of LLMs as cognitive models. This approach involves (i) leveraging computationally equivalent tasks that both an LLM and a rational agent need to master for solving a cognitive problem and (ii) examining the specific task distributions required for an LLM to exhibit human-like behaviors. We apply this approach to decision-making -- specifically risky and intertemporal choice -- where the key computationally equivalent task is the arithmetic of expected value calculations. We show that an LLM pretrained on an ecologically valid arithmetic dataset, which we call Arithmetic-GPT, predicts human behavior better than many traditional cognitive models. Pretraining LLMs on ecologically valid arithmetic datasets is sufficient to produce a strong correspondence between these models and human decision-making. Our results also suggest that LLMs used as cognitive models should be carefully investigated via ablation studies of the pretraining data.
Paper Structure (20 sections, 6 equations, 5 figures, 8 tables)

This paper contains 20 sections, 6 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Arithmetic-GPT: A Small Language Model Trained to Perform Arithmetic
  4. Model Details
  5. Synthetic Datasets
  6. Pretraining Details
  7. Human Targets
  8. Other Models
  9. Experimental Results
  10. Model Comparisons
  11. Implicit functions of probabilities, values, and discount factors
  12. Discussion
  13. Prompts
  14. Robustness Check
  15. Learning Curve
  16. ...and 5 more sections

Figures (5)

  • Figure 1: (A) Pre-training and evaluation pipelines. We began by generating a synthetic dataset comprised of mathematical equations including addition, subtraction, multiplication, and exponentiation. Arithmetic-GPT was pretrained on this synthetic dataset. After training, we froze model weights and extracted embeddings from the pretrained model, which then processes stylized choice tasks as input. These embeddings were subsequently compared with human choice data to evaluate their correspondence. (B) Ecological distributions of probabilities and values. In the top panel, English probability-describing phrases (black bars) can be modeled using a Beta$(0.27, 0.27)$ distribution. In the bottom panel, the value distribution of debits from UK bank accounts (scatterpoints) follows a power-law distribution. Figures adapted from zhu2020bayesian and stewart2006decision.
  • Figure 2: Embeddings from ecologically pretrained Arithmetic-GPT for inputs including (A) probabilities, (B) values, and (C) discount factors. Inputs are shown along the horizontal axes and embeddings are shown on the vertical axes. The embeddings, shown as black dots, were reduced to 1D using multidimensional scaling. Embeddings for probabilities and discount factors are normalized between 0 and 1 (see Appendix \ref{['ap:dimensionality_reduction']} for details). The red curves represent the best-fitting behavioral economic models: (A) the probability weighting function from PT with best-fitting $\gamma=0.58$, (B) the utility function from PT with best-fitting $\alpha=0.42, \beta=0.45, \lambda=1.4$, and (C) the hyperbolic discount function with best-fitting $k=0.08$.
  • Figure A1: (A) Training loss decreases over the course of training epochs for Arithmetic-GPT trained on ecological synthetic dataset. (B) Histogram displaying the differences between the top-1 responses of the pretrained Arithmetic-GPT model and the actual expected values in the ecological synthetic dataset.
  • Figure A2: Visualizations of embeddings from LLaMA-3-70B-Instruct.
  • Figure A3: Visualizations of embeddings from Arithmetic-GPTs pretrained on different synthetic datasets. (A) Uniform synthetic dataset. (B) Ablated uniform synthetic dataset. (C) Ablated ecological synthetic dataset.