Strategy Complexity of Reachability in Countable Stochastic 2-Player Games

TL;DR

This work analyzes the memory requirements of $\varepsilon$-optimal and optimal strategies in countably infinite two-player stochastic games with a reachability objective, distinguishing between concurrent and turn-based settings, finite versus infinite action sets, and uniformity across starts. Through a blend of lower-bound constructions (notably variants of the Big Match and nested BMI gadgets) and constructive upper-bound results (uniform $\varepsilon$-optimal strategies using a single public bit of memory, and inevitable infinite-memory needs under infinite Minimizer branching), the authors provide a comprehensive map of strategy complexity. Key findings show that $\varepsilon$-optimal Maximizer strategies can require infinite memory when Minimizer has infinite actions, while in finite-action regimes Maximizer can rely on compact public memory (often 1 bit) to achieve uniform $\varepsilon$-optimality; optimal strategies, when they exist, may still demand substantial memory, with precise bounds depending on turn-based versus concurrent structure and on whether random states are finitely or infinitely branching. The results advance understanding of memory usage in stochastic games and have implications for designing strategies in systems with countable state spaces and reachability objectives.

Abstract

We study countably infinite stochastic 2-player games with reachability objectives. Our results provide a complete picture of the memory requirements of $\varepsilon$-optimal (resp. optimal) strategies. These results depend on the size of the players' action sets and on whether one requires strategies that are uniform (i.e., independent of the start state). Our main result is that $\varepsilon$-optimal (resp. optimal) Maximizer strategies require infinite memory if Minimizer is allowed infinite action sets. This lower bound holds even under very strong restrictions. Even in the special case of infinitely branching turn-based reachability games, even if all states allow an almost surely winning Maximizer strategy, strategies with a step counter plus finite private memory are still useless. Regarding uniformity, we show that for Maximizer there need not exist positional (i.e., memoryless) uniformly $\varepsilon$-optimal strategies even in the special case of finite action sets or in finitely branching turn-based games. On the other hand, in games with finite action sets, there always exists a uniformly $\varepsilon$-optimal Maximizer strategy that uses just one bit of public memory.

Figures (11)

• Figure 1: The Concurrent Big Match on $\mathbb{Z}$; see \ref{['def:concurrent-big-match']}. On the right is a depiction of the game graph; on the left we see how joint actions from state $c_i$ are resolved.
• Figure 2: The Concurrent Big Match on $\mathbb{N}$; see \ref{['def:concurrent-big-match-zplus']}. On the right is a depiction of the game graph; on the left we see how joint actions from state $c_i$ are resolved.
• Figure 3: Example of sets $S_i[0]$ and $S_i[1]$ in the two layers of ${\mathcal{G}}$, where the outer and inner layers are $S\times \{1\}$ and $S \times \{0\}$, respectively.
• Figure 4: Example of a game $\bar{{\mathcal{G}}}_3$.
• Figure 5: Turn-based Big Match on $\mathbb{N}$.

Theorems & Definitions (77)

• Proposition 1
• Theorem 2
• Definition 1: Concurrent Big Match on $\mathbb{Z}$
• Theorem 3: FLS:1995, Theorem 1.1
• Definition 2: Concurrent Big Match on $\mathbb{N}$
• Theorem 4
• proof
• Lemma 5
• proof
• Theorem 6