Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation

TL;DR

This work tackles predicting human decisions in off-policy evaluation for language-based persuasion games by integrating real human-bot interaction data with a simulation-based data augmentation framework. The proposed approach uses a mixture of interpretable DM heuristics (Trustful, Language-based, Random) and an improving oracle to generate additional data, enabling robust OPE for unseen expert bots. Empirical results show that training with combined real and simulated data substantially improves prediction accuracy on hard instances (e.g., LSTM+S up by $7.1\%$ in the top 15% hardest cases) while maintaining performance on easier cases, with EF-based language representations outperforming embedding methods as simulation intensifies. The dataset of 87K decisions and the accompanying code are publicly released, underscoring the method's cost-effectiveness, interpretability, and potential applicability to other human-in-the-loop decision tasks in recommender systems and economic interactions.

Abstract

Recent advances in Large Language Models (LLMs) have spurred interest in designing LLM-based agents for tasks that involve interaction with human and artificial agents. This paper addresses a key aspect in the design of such agents: predicting human decisions in off-policy evaluation (OPE). We focus on language-based persuasion games, where an expert aims to influence the decision-maker through verbal messages. In our OPE framework, the prediction model is trained on human interaction data collected from encounters with one set of expert agents, and its performance is evaluated on interactions with a different set of experts. Using a dedicated application, we collected a dataset of 87K decisions from humans playing a repeated decision-making game with artificial agents. To enhance off-policy performance, we propose a simulation technique involving interactions across the entire agent space and simulated decision-makers. Our learning strategy yields significant OPE gains, e.g., improving prediction accuracy in the top 15% challenging cases by 7.1%. Our code and the large dataset we collected and generated are submitted as supplementary material and publicly available in our GitHub repository: https://github.com/eilamshapira/HumanChoicePrediction

Figures (14)

• Figure 1: Illustration of a single round in the language-based persuasion game. The bot expert starts by analyzing the interaction history from prior rounds (not depicted in the illustration) alongside a set of seven reviews, each consisting of a textual description and an associated score. Following a predefined strategy, it selects one review from the set and transmits only its textual content to the human Decision Maker (DM). The DM then evaluates the received review in the context of the full interaction history and chooses an action. In the final step, both the expert and the DM receive their payoffs, which are determined by the DM’s choice and the hotel's actual quality.
• Figure 2: A sample review from our hotel review dataset. The agent is exposed to both the textual part and the numerical rating of the review. The agent sends only the textual signal to the DM, which is not exposed to the numerical rating.
• Figure 3: An example strategy from the $E_\mathcal{B}$ set.
• Figure 4: Example of the update process of the temperament vector of a simulated-DM. Each simulated DM is assigned a nature vector, representing its inherent action probabilities. At the start of an interaction with a new agent, the DM’s temperament vector is initialized to that nature vector. In each round, the DM’s action is randomly chosen according to the probabilities in the temperament vector. After the round, the temperament vector is updated so that, with some positive probability, the likelihood of playing Oracle increases, while the probabilities of playing all other actions decrease.
• Figure 5: Performance of the models on different sets of hard (top) and easy (bottom) examples, with 95% bootstrap confidence intervals. Training on a combination of human-bot interaction and simulated data improves model performance on the hard sets without harming their performance on the easy sets.