Algorithms for zero-sum stochastic games with the risk-sensitive average criterion
Fang Chen, Xianping Guo, Xin Guo, Junyu Zhang
TL;DR
This work addresses computing the value and saddle points for zero-sum, finite-state, finite-action stochastic games under the risk-sensitive average criterion. It introduces an irreducibility coefficient $\gamma$ and an operator framework $\mathcal{L}$ to establish the existence of a value via the Shapley equation, with $J^*(i)=\theta^{-1}\ln \lambda_*$ and $\lambda_*$ obtained as the spectral growth rate of $\mathcal{L}$. The authors develop two concrete algorithms: one producing $\varepsilon$-approximations of the value through monotone sequences $\lambda_n$ and $\zeta_n$, and another delivering $\varepsilon$-saddle points by constructing tight bounds around $h^*$ (via $U_n^\varepsilon$) and solving per-state LPs to yield mini-max selectors. A smart-grid energy-management example demonstrates applicability, providing explicit policies and numerical convergence guarantees. Overall, the paper delivers first-principles, computable procedures for risk-sensitive saddle-point computation in finite-state stochastic games with rigorous convergence guarantees.
Abstract
This paper is an attempt to compute the value and saddle points of zero-sum risk-sensitive average stochastic games. For the average games with finite states and actions, we first introduce the so-called irreducibility coefficient and then establish its equivalence to the irreducibility condition. Using this equivalence,we develop an iteration algorithm to compute $\varepsilon$-approximations of the value (for any given $\varepsilon>0$) and show its convergence. Based on $\varepsilon$-approximations of the value and the irreducibility coefficient, we further propose another iteration algorithm, which is proved to obtain $\varepsilon$-saddle points in finite steps. Finally, a numerical example of energy management in smart grids is provided to illustrate our results.