Identifying Latent Intentions via Inverse Reinforcement Learning in Repeated Linear Public Good Games

TL;DR

This paper addresses the unexplained volatility observed in repeated linear public good games by introducing two methodological advances: Dynamic Time Warping (DTW) based clustering to identify temporally misaligned behavioral trajectories, and a Hierarchical Inverse Q-Learning (HIQL) framework that models discrete switches between latent cooperative and defective intentions. Applying these methods to a large dataset of $50{,}390$ decisions from $2{,}938$ participants, the authors uncover six behavioral clusters, including a novel Switchers type that accounts for about $21.4\%$ of participants and frequently reverses intentions. The HIQL model integrates observed actions and state information to infer latent intentions, revealing both stable patterns (persistent free riders, unconditional and consistent cooperators) and switching dynamics (Switchers) that are explanations for previously ambiguous residuals. Recognizing intentional volatility demonstrates how forward-looking partners can forgive transient defections, aiding strategies to sustain cooperation, with broad implications for organizational behavior and financial decision-making. The approach is highly transferable to other time-series domains, offering a data-driven, interpretable framework for identifying latent intentions and their transitions in complex social and economic settings.

Abstract

Behavior in repeated public goods games continues to challenge standard theory: heterogeneous social preferences can explain first-round contributions, but not the substantial volatility observed across repeated interactions. Using 50,390 decisions from 2,938 participants, we introduce two methodological advances to address this gap. First, we cluster behavioral trajectories by their temporal shape using Dynamic Time Warping, yielding distinct and theoretically interpretable behavioral types. Second, we apply a hierarchical inverse Q-learning framework that models decisions as discrete switches between latent cooperative and defective intentions. This approach reveals a large (21.4%) and previously unmodeled behavioral type -- Switchers -- who frequently reverse intentions rather than commit to stable strategies. At the same time, the framework recovers canonical strategic behaviors such as persistent cooperation and free-riding. Substantively, recognizing intentional volatility helps sustain cooperation: brief defections by Switchers often reverse, so strategic patience can prevent unnecessary breakdowns.

Figures (15)

• Figure 1: Conceptual Illustration of Euclidean and DTW distances.(Left) Stacked sinusoidal curves represent two players with similar but vertically offset contribution patterns. (Center) Horizontally shifted curves illustrate temporal misalignment between players’ actions; dotted vertical lines denote pairwise Euclidean distances. (Right) DTW alignment paths (black solid lines) show the optimal nonlinear correspondence between the two time series, compensating for temporal shifts.
• Figure 2: Global vs. Local Alignment.Participants are clustered based on two different distance metrics: DTW (global) and Euclidean (local) distance. Each small panel represents one cluster within a method, with participants sorted by their average contribution (highest on top). Red dots indicate when the most substantial consecutive drop in contribution occurs per participant. In the DTW cluster 2, a quadratic regression curve is overlaid to illustrate that the drop can occur at any point in the game. In the Euclidean clusters 2, 3, and 4, a vertical reference line is drawn to mark the period where most detected decline points are concentrated. Cell color encodes contribution magnitude (yellow indicates higher relative contribution, dark blue lower), and sample sizes per cluster appear above each panel.
• Figure 3: Clustering Analysis of Action-State Patterns and Intention Dynamics. (1) Heatmaps of action and state trajectories by cluster, ordered by mean action. Clustering uses two-dimensional time series of own contributions (action) and others’ average contributions (state), normalized to [0,1]. Rows denote individuals (UIDs), and rows as well as clusters are ordered from lowest to highest average contribution. (2) DTW barycenters of action-state patterns per cluster, with interquartile ranges shaded. (3) DTW barycenter averages of posterior intention adoption probabilities by cluster, with interquartile ranges shaded. Two intentions are estimated; the second is the complement of the first. Values closer to one indicate a higher likelihood that the intention is adopted. (5) Average transition probabilities between cooperative and defecting intentions per cluster.
• Figure 4: High-Switcher Identification Through Multi-Metric Behavioral Classification. (A) Classification of all 24 behavioral clusters across four game lengths (7, 10, 20, 30 rounds) in a two-dimensional space defined by stickiness (lag-1 autocorrelation) and switching rate (mean transition probability between latent cooperative and defective intentions). Switchers (stars) meet all three criteria. Dashed red lines show decision boundaries. (B--D) Cluster-level metrics with interquartile ranges (error bars). (B) Switchers exhibit near-zero stickiness. (C) Switching rates for Switchers (39--48%) approach random-walk behavior ($50\%$), indicating frequent reversals in latent intention. (D) Posterior volatility reflects how often inferred intentions cross the $0.5$ threshold. Only Switchers pair this with low stickiness and high switching.
• Figure S.1: Visualization of the Raw Data.The y-axis represents participant identifiers (UIDs), while the x-axis tracks the rounds played. Color intensity indicates the size of contributions, normalized between 0 and 1. The panels labeled 'Contribution' reflect each player's own contribution. The panels labeled 'Experience' represent the average contribution of the other players in the group during the preceding round. Within subplot-pairs, UIDs are sorted by their average contribution. Notably, there is heterogeneity with respect to the number of rounds played: The data is divided into subsets with games lasting 7, 10, 20, or 30 rounds. Furthermore, there exists heterogeneity concerning game parameters. Most notably, one study played a 7-round game (first column) with an exceptionally large group size of 100, resulting in unusually homogeneous average contributions.