Inferring Strategies from Observations in Long Iterated Prisoner's Dilemma Experiments

Abstract

While many theoretical studies have revealed the strategies that could lead to and maintain cooperation in the Iterated Prisoner's Dilemma, less is known about what human participants actually do in this game and how strategies change when being confronted with anonymous partners in each round. Previous attempts used short experiments, made different assumptions of possible strategies, and led to very different conclusions. We present here two long treatments that differ in the partner matching strategy used, i.e. fixed or shuffled partners. Here we use unsupervised methods to cluster the players based on their actions and then Hidden Markov Model to infer what are those strategies in each cluster. Analysis of the inferred strategies reveals that fixed partner interaction leads to a behavioral self-organization. Shuffled partners generate subgroups of strategies that remain entangled, apparently blocking the self-selection process that leads to fully cooperating participants in the fixed partner treatment. Analyzing the latter in more detail shows that AllC, AllD, TFT- and WSLS-like behavior can be observed. This study also reveals that long treatments are needed as experiments less than 25 rounds capture mostly the learning phase participants go through in these kinds of experiments.

Figures (15)

• Figure 1: Probability of each context and the cooperation rate per treatment. In the x-axis, the context of each decision is shown, i.e. the actions of the previous round. The green bar in each chart shows the fraction of subsequent cooperative actions when experiencing the particular context. This way, in FP (panel A), there are many rounds with context CC (mutual cooperation) and DD(mutual defection). In SP (panel B) DD occurs clearly more often.
• Figure 2: Probability of cooperation given each context per cluster (using K-Means). A) Top row shows the context composition of each cluster in the FP treatment, the bottom row does the same for each SP cluster. In the FP treatment, clusters A and B are composed of opposing experiences, either $CC$ for A or $DD$ for B. Cluster C is a mix with a bias towards mutually cooperative encounters. In the SP treatment, bottom row, the experiences tend to be more of the $DD$ kind, as shown in clusters D and E. Cluster F in SP showed a mixed experience, yet with a different context composition as C. B) tSNE visualization of the different clusters A-F. As can be seen, cluster A and B lie at the two extremes of this embedding. The clusters D and E have more in common with the defective behaviors in B than the cooperative ones, i.e. A and C. C differentiates itself from the defective clusters as $CC$ is more likely to occur, yet part of C overlaps with the behaviors observed in F.
• Figure 3: Probability of cooperation given each context per cluster for the FP treatment. In each plot, the conditional probability of cooperating for each context is given. The bars represent the binomial error. Each contextual cluster is divided into sub-clusters that were identified using the frequency of cooperation given a context. It can be seen that in cluster A (left) that the differences between the sub-clusters are due to the difference in cooperation for the context $DD$. In cluster B (center), they all have a similar response to the $DD$ context, which explains the distribution seen in Figure \ref{['fig:context_cluster_chart']}. Cluster C (right) shows a mixed experience among the sub-clusters, with cluster C.0 showing significantly stronger cooperative response in case of the context $CD$.
• Figure 4: Hidden Markov Models for the FP treatment. Here, the eight sub-clusters found in FP have a HMM that describes each sub-cluster's strategy for the data over all rounds. Bold rectangles represent the initial state, while the others represent subsequent hidden states. Symbols with a probability lower than 0.05 are not shown, as well as the transition probability between states lower or equal to 0.01. For example, in sub-cluster A.0, the subjects play $D$ in the context $CD$ with a very small probability to continue, but a big one to pass to the other hidden state in the sequence, where they mutually cooperated ($(CC)C$). On the other hand, sub-cluster A.1 has some probability to defect given mutual defection $(DD)D$, which A.0 does not show. Overall, it appears a model with only one state is preferred by the training method (see Methods).
• Figure 5: Probability of cooperation given each context for the Shuffled Partners treatment. In the x-axis, it is shown the probability of cooperating given that in the previous round there was a certain context. The bars represent the binomial error. Each cluster is divided into sub-cluster that were divided using the frequency of cooperation given a context. As in FP (Figure \ref{['fig:prob_coop_fix']}), the sub-clusters present differences even when in their clusters a certain context dominates. For example, in p($C|CC$), it is shown how sub-cluster E.2 has a very low probability to cooperate given cooperation in the previous round, while the other sub-clusters are situated between 0.4 and 0.6.