Capturing the Complexity of Human Strategic Decision-Making with Machine Learning

TL;DR

A large-scale study of strategic decision-making in the context of initial play in two-player matrix games, analysing over 90,000 human decisions across more than 2,400 procedurally generated games that span a much wider space than previous datasets shows that a deep neural network trained on this dataset predicts human choices with greater accuracy.

Abstract

Understanding how people behave in strategic settings--where they make decisions based on their expectations about the behavior of others--is a long-standing problem in the behavioral sciences. We conduct the largest study to date of strategic decision-making in the context of initial play in two-player matrix games, analyzing over 90,000 human decisions across more than 2,400 procedurally generated games that span a much wider space than previous datasets. We show that a deep neural network trained on these data predicts people's choices better than leading theories of strategic behavior, indicating that there is systematic variation that is not explained by those theories. We then modify the network to produce a new, interpretable behavioral model, revealing what the original network learned about people: their ability to optimally respond and their capacity to reason about others are dependent on the complexity of individual games. This context-dependence is critical in explaining deviations from the rational Nash equilibrium, response times, and uncertainty in strategic decisions. More broadly, our results demonstrate how machine learning can be applied beyond prediction to further help generate novel explanations of complex human behavior.

Figures (9)

• Figure 1: Matrix games.(a) An example game interface presented to participants, who acted as the row player in each $2 \times 2$ game. The blue numbers represent the payoffs for the row player, who chooses between strategies $A$ and $B$, while the red numbers represent the payoffs for the column player, who chooses between strategies $C$ and $D$. (b) Visualization of game space. Each game is uniquely represented by an 8-integer vector, corresponding to the payoffs to the two players under different configurations of choices. We used $t$-distributed stochastic neighbor embeddings van2008visualizing to visualize the spatial relationship between games in a 2D plot, using the Euclidean distance between the embeddings of our best-performing neural network model. Points represent individual games. The colors represent the game topology specific to the row player following robinson2005topology.
• Figure 2: Model comparisons.(a) The context-invariant level-$k$ quantal-response model involves three parameters that do not vary across games: strategic sophistication (i.e., $k$), the players' noisiness (i.e., $\eta_\text{self}$) and risk aversion. (b) A Multi-layer Perceptron (MLP) model directly uses the game matrix as input to estimate choice probabilities, without imposing any specific game-theoretic decision-making structure. (c) The level-$k$ neural quantal-response model is a context-dependent model allowing the $\eta_\text{self}$ parameter to vary across games. It employs an MLP model, which uses the game matrix as input to estimate game-specific $\eta_\text{self}$. (d) The level-$k$ neural quantal-response and neural belief noise model extends the model in (c) by further learning the game-specific $\eta^s_\text{other}$ and $k$ parameters through two MLP models, each of which takes the game matrix as input. (e) Context-dependent models, incorporating at least one neural network component that allows some or all model parameters to vary across games, outperform context-invariant models in terms of completeness. Higher completeness indicates greater predictive accuracy for human behaviors, with 100% completeness matching the predictive accuracy of the MLP model. All reported results were based on 10-fold cross-validation (see Supplementary Information for details). Our focus was on the heterogeneity across games rather than the heterogeneity across participants.
• Figure 3: Developing an interpretable complexity index for strategic games.(a) To construct an index of game complexity, we use LASSO regressions to learn game features that correlate with the game-specific $\eta_\text{self}$ parameter that is estimated by the MLP in the Level-2 Neural QR and Neural Belief Noise model. (b) The psychometric functions illustrate the relationship between the expected utility differences of two strategies and the proportion of choices for strategy $A$. Expected utility was calculated under the assumption of a level-1 player. Red (blue) dots represent high (low) complexity games, determined by a median split on the complexity index. Error bars represent $\pm$SE. (c) The same psychometric function and effect of complexity was found in the followup experiment. (d) The complexity index shows a statistically significant correlation with response times (RTs) in the games of the main experiment. (e) The complexity index generalized to the followup experiment and demonstrated statistically significant correlations with both RTs and cognitive uncertainty ratings.
• Figure S1: Example of a game matrix: The payoff matrix for the row player ('self') is in blue, while the payoff matrix for the column player ('other') is in red.
• Figure :