Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Time-Dependent Blackwell Approachability and Application to Absorbing Games

Joon Kwon, Yijun Wan, Bruno Ziliotto

TL;DR

The paper extends Blackwell's approachability to time-dependent outcomes and inner products, and proves a general convergence guarantee for the time-varying Blackwell algorithm. It shows that when the target is an orthant, a family of time-dependent inner products yields coordinate-wise convergence rates for the average payoff. The authors apply this framework to absorbing games, constructing ε-uniformly optimal strategies via an auxiliary two-dimensional approachability problem. This work links online learning tools with stochastic game theory, offering a self-contained alternative to classical proofs and suggesting broader applications to regret minimization and other sequential decision problems.

Abstract

Blackwell's approachability (Blackwell, 1954, 1956) is a very general online learning framework where a Decision Maker obtains vector-valued outcomes, and aims at the convergence of the average outcome to a given ``target'' set. Blackwell gave a sufficient condition for the decision maker having a strategy guaranteeing such a convergence against an adversarial environment, as well as what we now call the Blackwell's algorithm, which then ensures convergence. Blackwell's approachability has since been applied to numerous problems, in regret minimization and game theory, in particular. We extend this framework by allowing the outcome function and the inner product to be time-dependent. We establish a general guarantee for the natural extension to this framework of Blackwell's algorithm. In the case where the target set is an orthant, we present a family of time-dependent inner products which yields different convergence speeds for each coordinate of the average outcome. We apply this framework to absorbing games (an important class of stochastic games) for which we construct $\varepsilon$-uniformly optimal strategies using Blackwell's algorithm in a well-chosen auxiliary approachability problem, thereby giving a novel illustration of the relevance of online learning tools for solving games.

Time-Dependent Blackwell Approachability and Application to Absorbing Games

TL;DR

The paper extends Blackwell's approachability to time-dependent outcomes and inner products, and proves a general convergence guarantee for the time-varying Blackwell algorithm. It shows that when the target is an orthant, a family of time-dependent inner products yields coordinate-wise convergence rates for the average payoff. The authors apply this framework to absorbing games, constructing ε-uniformly optimal strategies via an auxiliary two-dimensional approachability problem. This work links online learning tools with stochastic game theory, offering a self-contained alternative to classical proofs and suggesting broader applications to regret minimization and other sequential decision problems.

Abstract

Blackwell's approachability (Blackwell, 1954, 1956) is a very general online learning framework where a Decision Maker obtains vector-valued outcomes, and aims at the convergence of the average outcome to a given ``target'' set. Blackwell gave a sufficient condition for the decision maker having a strategy guaranteeing such a convergence against an adversarial environment, as well as what we now call the Blackwell's algorithm, which then ensures convergence. Blackwell's approachability has since been applied to numerous problems, in regret minimization and game theory, in particular. We extend this framework by allowing the outcome function and the inner product to be time-dependent. We establish a general guarantee for the natural extension to this framework of Blackwell's algorithm. In the case where the target set is an orthant, we present a family of time-dependent inner products which yields different convergence speeds for each coordinate of the average outcome. We apply this framework to absorbing games (an important class of stochastic games) for which we construct -uniformly optimal strategies using Blackwell's algorithm in a well-chosen auxiliary approachability problem, thereby giving a novel illustration of the relevance of online learning tools for solving games.
Paper Structure (18 sections, 12 theorems, 67 equations)

This paper contains 18 sections, 12 theorems, 67 equations.

Table of Contents

  1. Introduction
  2. Contributions and Summary
  3. Related Work
  4. Notation
  5. A Generalized Approachability Framework
  6. Analysis
  7. Absorbing Games
  8. Definition
  9. Values
  10. Balanced Strategies and Lower Bounds on the Average Payoff
  11. An Auxiliary Approachability Problem
  12. The Resulting Strategy for Player I in the Absorbing Game
  13. Comparison with Other Proofs of Theorem \ref{['theo:main']}
  14. Conclusion and Perspectives
  15. On General Closed Convex Targets
  16. ...and 3 more sections

Key Result

Proposition 2.6

Assume that $\mathcal{A}$ and $\mathcal{B}$ are convex sets of finite dimensional vectors spaces, such that $\mathcal{A}$ is compact and all outcome functions $\rho\in \mathcal{R}$ are bi-affine. Then, a closed convex cone $\mathcal{C}\subset \mathbb{R}^d$ satisfies Blackwell's condition with respec

Theorems & Definitions (31)

  • Definition 2.1
  • Remark 2.2
  • Definition 2.3: Blackwell's condition
  • Definition 2.4: Blackwell's algorithm
  • Remark 2.5
  • Proposition 2.6: Blackwell's dual condition
  • proof
  • Theorem 3.1
  • proof
  • Remark 3.2
  • ...and 21 more