Different Strokes in Randomised Strategies: Revisiting Kuhn's Theorem under Finite-Memory Assumptions

TL;DR

This work provides a complete taxonomy of the classes of finite-memory strategies obtained by varying which of the three aforementioned components are randomised, and holds in games of perfect and imperfect information with perfect recall, and in games with more than two players.

Abstract

Two-player (antagonistic) games on (possibly stochastic) graphs are a prevalent model in theoretical computer science, notably as a framework for reactive synthesis. Optimal strategies may require randomisation when dealing with inherently probabilistic goals, balancing multiple objectives, or in contexts of partial information. There is no unique way to define randomised strategies. For instance, one can use so-called mixed strategies or behavioural ones. In the most general setting, these two classes do not share the same expressiveness. A seminal result in game theory -- Kuhn's theorem -- asserts their equivalence in games of perfect recall. This result crucially relies on the possibility for strategies to use infinite memory, i.e., unlimited knowledge of all past observations. However, computer systems are finite in practice. Hence it is pertinent to restrict our attention to finite-memory strategies, defined as automata with outputs. Randomisation can be implemented in these in different ways: the initialisation, outputs or transitions can be randomised or deterministic respectively. Depending on which aspects are randomised, the expressiveness of the corresponding class of finite-memory strategies differs. In this work, we study two-player concurrent stochastic games and provide a complete taxonomy of the classes of finite-memory strategies obtained by varying which of the three aforementioned components are randomised. Our taxonomy holds in games of perfect and imperfect information with perfect recall, and in games with more than two players. We also provide an adapted taxonomy for games with imperfect recall.

Figures (10)

• Figure 1.1: Lattice of strategy classes in terms of expressible probability distributions over plays against all strategies of the other player. In the three-letter acronyms, the letters, in order, refer to the initialisation, outputs and updates of the Mealy machines: D and R respectively denote deterministic and randomised components. Each line in the figure indicates that the class above is strictly more expressive than the class below.
• Figure 4.1: The game $\mathcal{G}_3$ from the proof of Lemma \ref{['lemma:mixed:behavioural:bound']}. Circles and squares respectively represent states controlled by $\mathcal{P}_{1}$ and $\mathcal{P}_{2}$.
• Figure 4.2: Representation of cumulative probability of actions under strategy $\mathcal{M}$ and derived memoryless strategies.
• Figure 5.1: The MDP $\mathcal{G}_{a, b}$ with a single state and two actions.
• Figure 5.2: Depictions of Mealy machines witnessing the strictness of three inclusions asserted in Figure \ref{['figure:lattice']}. For the sake of readability, we do not label transitions by $s$ as it is the sole state the Mealy machines can read in $\mathcal{G}_{a, b}$, and the only state with a choice in the games of Figure \ref{['figure:strictness:games']}.

Theorems & Definitions (34)

• lemma 1: Strategic criterion for outcome-equivalence
• proof
• theorem 1
• proof
• lemma 2
• proof
• theorem 2
• proof
• theorem 3
• proof