Predicting Human Choice Between Textually Described Lotteries
Eyal Marantz, Ori Plonsky
TL;DR
This paper tackles predicting human choices when option descriptions are textual rather than numeric by introducing TextualChoices-1K and evaluating four modeling approaches that leverage LLMs, embeddings, and theory-guided features.It shows fine-tuned GPT-4o delivers the best performance on textual descriptions, surpassing theory-based hybrids that excel in numeric tasks, and reveals a fundamental gap between textual and numeric decision processes.The study demonstrates cross-domain insights: neural models can leverage language cues effectively for textual risk decisions, while traditional BEAST-based hybrids remain strong for numerically described lotteries, underscoring the need for task-specific modeling strategies.Overall, the findings motivate developing hybrid models that bridge textual interpretation with behavioral theory to better predict real-world decisions described in natural language.
Abstract
Predicting human decision-making under risk and uncertainty is a long-standing challenge in cognitive science, economics, and AI. While prior research has focused on numerically described lotteries, real-world decisions often rely on textual descriptions. This study conducts the first large-scale exploration of human decision-making in such tasks using a large dataset of one-shot binary choices between textually described lotteries. We evaluate multiple computational approaches, including fine-tuning Large Language Models (LLMs), leveraging embeddings, and integrating behavioral theories of choice under risk. Our results show that fine-tuned LLMs, specifically GPT-4o, outperform hybrid models that incorporate behavioral theory, challenging established methods in numerical settings. These findings highlight fundamental differences in how textual and numerical information influence decision-making and underscore the need for new modeling strategies to bridge this gap.