Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonstationary nonzero-sum Markov games under a probability criterion

Xin Guo, Xin Wen

TL;DR

The paper addresses nonstationary $N$-person nonzero-sum stochastic games under a probability criterion, where both transitions and rewards evolve with time. It develops a sequence of optimality equations and a mild reachability condition, proves a comparison theorem, and establishes the existence of Nash equilibria for history-dependent policies. An efficient algorithm is provided to compute $\epsilon$-Nash equilibria with finite-time guarantees, and the approach is demonstrated on a nonstationary energy-management example and a numerical insurance-market scenario. The results extend prior stationary analyses to nonstationary settings, offering both theoretical guarantees and practical computation for reliable, probability-based performance criteria in dynamic multi-agent systems.

Abstract

This paper deals with N-person nonzero-sum discrete-time Markov games under a probability criterion, in which the transition probabilities and reward functions are allowed to vary with time. Differing from the existing works on the expected reward criteria, our concern here is to maximize the probabilities that the accumulated rewards until the first passage time to any target set exceed a given goal, which represent the reliability of the players income. Under a mild condition, by developing a comparison theorem for the probability criterion, we prove the existence of a Nash equilibrium over history-dependent policies. Moreover, we provide an efficient algorithm for computing epsilon-Nash equilibria. Finally, we illustrate our main results by a nonstationary energy management model and take a numerical experiment.

Nonstationary nonzero-sum Markov games under a probability criterion

TL;DR

The paper addresses nonstationary -person nonzero-sum stochastic games under a probability criterion, where both transitions and rewards evolve with time. It develops a sequence of optimality equations and a mild reachability condition, proves a comparison theorem, and establishes the existence of Nash equilibria for history-dependent policies. An efficient algorithm is provided to compute -Nash equilibria with finite-time guarantees, and the approach is demonstrated on a nonstationary energy-management example and a numerical insurance-market scenario. The results extend prior stationary analyses to nonstationary settings, offering both theoretical guarantees and practical computation for reliable, probability-based performance criteria in dynamic multi-agent systems.

Abstract

This paper deals with N-person nonzero-sum discrete-time Markov games under a probability criterion, in which the transition probabilities and reward functions are allowed to vary with time. Differing from the existing works on the expected reward criteria, our concern here is to maximize the probabilities that the accumulated rewards until the first passage time to any target set exceed a given goal, which represent the reliability of the players income. Under a mild condition, by developing a comparison theorem for the probability criterion, we prove the existence of a Nash equilibrium over history-dependent policies. Moreover, we provide an efficient algorithm for computing epsilon-Nash equilibria. Finally, we illustrate our main results by a nonstationary energy management model and take a numerical experiment.
Paper Structure (6 sections, 8 theorems, 70 equations, 1 figure, 2 tables)

This paper contains 6 sections, 8 theorems, 70 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. The game model
  3. Preliminaries
  4. The existence of a Nash equilibrium
  5. An Algorithm
  6. Examples

Key Result

Proposition 1

For each $n\ge 0$, if there exists a constant $\beta_n\in[0,1)$ such that $p_n(D|i,\bm{a})\ge \beta_n$ for all $i\in S_n\setminus D$, $\bm{a}\in \bm{A}_n(i)$ and $\sum_{n=0}^{\infty}\beta_n=+\infty$, then Assumption A holds.

Figures (1)

  • Figure 1: The number of iterations of Algorithm A with respect to $\epsilon$.

Theorems & Definitions (15)

  • Definition 1
  • Definition 2
  • Proposition 1
  • Remark 1
  • Theorem 1
  • Lemma 1
  • Theorem 2
  • Lemma 2
  • Definition 3
  • Theorem 3
  • ...and 5 more