The Economics of No-regret Learning Algorithms

TL;DR

The article surveys the economics of no-regret learning algorithms, connecting classical online learning to outcomes in repeated strategic interactions. It foregrounds how no-regret dynamics converge to correlated or coarse correlated equilibria, and analyzes the robustness of these outcomes to manipulation through swap regret. The text develops foundational algorithms (EW, FTL, PFTL) and their bandit extensions (Exp3) and explains reductions that transfer guarantees from full feedback to partial feedback, with implications for repeated games, bidding, and regulation. It also discusses inference about bidders’ values from observed history and proposes data-driven regulatory frameworks based on swap-regret constraints to curb algorithmic collusion while preserving innovation. Overall, the work links no-regret guarantees to tractable equilibrium concepts and to practical policy tools for algorithmic markets.

Abstract

A fundamental challenge for modern economics is to understand what happens when actors in an economy are replaced with algorithms. Like rationality has enabled understanding of outcomes of classical economic actors, no-regret can enable the understanding of outcomes of algorithmic actors. This review article covers the classical computer science literature on no-regret algorithms to provide a foundation for an overview of the latest economics research on no-regret algorithms, focusing on the emerging topics of manipulation, statistical inference, and algorithmic collusion.

Figures (6)

• Figure 2.1: Illustration of \ref{['ex:btl-vs-opt']} and \ref{['thm:btl-ge-opt']}. Payoff for each of the actions are depicted as the width of rectangles numbered by their round. In each round $i$, the payoffs $\mathop{\mathrm{btl}}\nolimits^i$ selected by $\mathop{\mathrm{BTL}}\nolimits$ are striped, the payoffs $\mathop{\mathrm{opt}}\nolimits^i$ accumulated by $\mathop{\mathrm{OPT}}\nolimits$ are shaded gray, and we have $\mathop{\mathrm{btl}}\nolimits^i \geq \mathop{\mathrm{opt}}\nolimits^i$.
• Figure 2.2: The coupling argument of \ref{['lem:stability']} depicted. The coin flips depicted in the center figure are Tails; Heads; Tails; Heads; Tails; Heads (as numbered 1--6 in the figure). Coins that flip Heads are dashed gray. The coin flip in the right figure is Tails (numbered 7). Depicted on the right, for these coin flips in round $i=4$, $\mathop{\mathrm{PFTL}}\nolimits$ and $\mathop{\mathrm{PBTL}}\nolimits$ obtain the same payoff.
• Figure 3.1: Reduction from multi-armed bandit learning to full-feedback online learning.
• Figure 4.1: Reduction from swap regret to best-in-hindsight regret.
• Figure 7.1: Rationalizable set for bidder $j$ defined by linear inequalities. The minimum rationalizable regret gives the value-regret pair $(\tilde{v}_{j},\tilde{r}_{j})$.

Theorems & Definitions (68)

• Definition 2.1: Follow the Leader, $\mathop{\mathrm{FTL}}\nolimits$
• Example 2.1
• Theorem 2.1
• proof
• Definition 2.2: Exponential Weights, $\mathop{\mathrm{EW}}\nolimits$
• Example 2.2
• Theorem 2.2
• Corollary 2.1
• proof
• Definition 2.3: Be the Leader, $\mathop{\mathrm{BTL}}\nolimits$