Prospect Theory Fails for LLMs: Revealing Instability of Decision-Making under Epistemic Uncertainty

TL;DR

This study probes whether Prospect Theory can explain Large Language Model decision-making under epistemic uncertainty expressed via linguistic markers. It introduces a three-stage evaluation that first estimates PT parameters from precise probabilities, then maps epistemic markers to numerical probabilities, and finally tests decision-making with marker-based uncertainties. The results show inconsistent PT fits across models, divergent marker mappings, and instability of PT parameters under linguistic uncertainty, suggesting that human-centric decision theories may not robustly predict LLM behavior. The work underscores the need for calibrated epistemic signaling and possibly alternative frameworks for modeling LLM risk under uncertainty, and provides an open-source codebase to support replication and extension.

Abstract

Prospect Theory (PT) models human decision-making under uncertainty, while epistemic markers (e.g., maybe) serve to express uncertainty in language. However, it remains largely unexplored whether Prospect Theory applies to contemporary Large Language Models and whether epistemic markers, which express human uncertainty, affect their decision-making behaviour. To address these research gaps, we design a three-stage experiment based on economic questionnaires. We propose a more general and precise evaluation framework to model LLMs' decision-making behaviour under PT, introducing uncertainty through the empirical probability values associated with commonly used epistemic markers in comparable contexts. We then incorporate epistemic markers into the evaluation framework based on their corresponding probability values to examine their influence on LLM decision-making behaviours. Our findings suggest that modelling LLMs' decision-making with PT is not consistently reliable, particularly when uncertainty is expressed in diverse linguistic forms. Our code is released in https://github.com/HKUST-KnowComp/MarPT.

Figures (9)

• Figure 1: An overview of our three-stage experiment. Stage 1 fits PT parameters from binary choices with precise probabilities. Stage 2 estimates each marker’s internal probability from the point where options K and U are chosen equally. Stage 3 substitutes probabilities with markers to measure the effect of linguistic uncertainty markers.
• Figure 2: Visual illustration of the three PT parameters ($\sigma$, $\lambda$, $\gamma$). Each parameter is shown with its meaning and an interpretation of its directional significance.
• Figure 3: An illustration of calculating $p_{mapping}$. We use linear interpolation for fitting and calculate $p_{mapping}$ through the similarity ratio.
• Figure 4: Probability mapping of different models for different markers. While different models assign divergent probability values to the same marker, the relative ordering of markers remains broadly consistent.
• Figure 5: The PT parameters estimated across rounds. The "Baseline" represents results from the first stage of our experiment. In subsequent rounds, parameters fluctuate relative to the baseline, indicating instability in the model’s PT alignment under epistemic perturbations.