Betting on Equilibrium: Monitoring Strategic Behavior in Multi-Agent Systems

TL;DR

The paper tackles real-time monitoring of equilibrium behavior in multi-agent systems by introducing a sequential, anytime-valid testing framework built on e-values and test supermartingales. It unifies Nash, correlated, and coarse correlated equilibria, and extends from repeated normal-form games to stochastic games with state-dependent dynamics. By pairing online FWER and FDR control (via mixture martingales and e-BH procedures) with likelihood-ratio testing in dynamic environments, the approach provides finite-time guarantees and interpretable metrics for departures from equilibrium. The framework is validated through experiments in normal-form and grid-based stochastic settings, demonstrating sharp detection power, robustness to unknown deviations, and asymptotic scaling laws of detection time. This work offers a principled toolkit for safety, reliability, and compliance monitoring in complex, adaptive multi-agent systems, with practical impact for automated decision-making and autonomous coordination.

Abstract

In many multi-agent systems, agents interact repeatedly and are expected to settle into equilibrium behavior over time. Yet in practice, behavior often drifts, and detecting such deviations in real time remains an open challenge. We introduce a sequential testing framework that monitors whether observed play in repeated games is consistent with equilibrium, without assuming a fixed sample size. Our approach builds on the e-value framework for safe anytime-valid inference: by "betting" against equilibrium, we construct a test supermartingale that accumulates evidence whenever observed payoffs systematically violate equilibrium conditions. This yields a statistically sound, interpretable measure of departure from equilibrium that can be monitored online. We also leverage Benjamini-Hochberg-type procedures to increase detection power in large games while rigorously controlling the false discovery rate. Our framework unifies the treatment of Nash, correlated, and coarse correlated equilibria, offering finite-time guarantees and a detailed analysis of detection times. Moreover, we extend our method to stochastic games, broadening its applicability beyond repeated-play settings.

Figures (10)

• Figure 1: Comparison of FWER and FDR detection performance over 300 runs in the $2\times2$ normal-form game. The FDR procedure consistently detects deviations earlier. (Top-Left) Distribution of stopping times. (Top-Right) Cumulative detection rate over time. (Bottom-Left) Frequency of detection for each specific hypothesis, showing the procedure correctly targets the two true alternative signals. (Bottom-Right) Run-by-run scatter plot.
• Figure 2: Asymptotic scaling in Grid Soccer. Empirical detection times (blue) align with the theoretical $O(1/\varepsilon^2)$ curve (green), confirming optimal scaling, with both curves anchored at the final point to compare asymptotic scaling independent of constants.
• Figure 3: Robustness of the mixture martingale. The mixture detector achieves the same optimal $O(1/\varepsilon^2)$ scaling.
• Figure 4: Evolution of all four test supermartingales across 300 independent runs under the null hypothesis $\mathcal{H}_0$. The horizontal dotted line is the FWER rejection threshold $b=20$. The empirical FWER (0.01) is well below the target level $\alpha=0.2$.
• Figure 5: Six sample trajectories from the $\mathcal{H}_1$ experiment ($\alpha=0.2$, $\lambda=0.05$). Each plot shows the two true supermartingales $M_{1,t}^{(0)}$ and $M_{2,t}^{(0)}$ against the FWER ($b=20$, dashed black) and FDR $k=2$ ($b=10$, dotted green) thresholds. Vertical lines indicate stopping times.

Theorems & Definitions (64)

• Definition 2.1: Nash Equilibrium
• Lemma 2.2
• Lemma 2.3
• Proposition 2.4
• Corollary 2.5
• Theorem 2.6
• Theorem 2.7
• Theorem 2.8
• Proposition 2.9
• Remark 2.10