Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A forward algorithm for a class of Markov zero-sum stopping games

Nhat-Thang Le

TL;DR

We present a forward-type algorithm for a class of Markov zero-sum stopping games on finite state spaces, extending the one-player forward algorithm to two competing players. The method constructs a decreasing sequence of value functions $V_k$ and shrinking sets $S_k$ that converge in finite steps to the game's value $V$ while recovering the critical regions where $V$ hits $\\psi$ or $\\phi$; the termination bound is below $|E|^2$ steps. Theoretical guarantees ensure a Nash equilibrium and finite termination, and we illustrate the approach with computational examples on birth-death processes and random walks, highlighting the potential for multiple NE in certain constructions. The framework provides a practical tool for computing the value and understanding the structure of NE in Markov stopping games with wide-ranging potential applications in economics and finance.

Abstract

In this paper, we propose a new efficient algorithm to compute the value function for zero-sum stopping games featuring two players with opposing interests. This can be seen as a game version of the ''forward algorithm'' for (one-player) optimal stopping problem, first introduced by Irle [6] for discrete-time Markov processes and later revisited by Miclo \& Villeneuve [8] for continuous-time Markov processes on general state spaces. This paper focuses on a game driven by a homogeneous Markov process taking values in a finite state space and also discusses about the number of iterations needed. Illustrated computational implementations for a few particular examples are also provided.

A forward algorithm for a class of Markov zero-sum stopping games

TL;DR

We present a forward-type algorithm for a class of Markov zero-sum stopping games on finite state spaces, extending the one-player forward algorithm to two competing players. The method constructs a decreasing sequence of value functions and shrinking sets that converge in finite steps to the game's value while recovering the critical regions where hits or ; the termination bound is below steps. Theoretical guarantees ensure a Nash equilibrium and finite termination, and we illustrate the approach with computational examples on birth-death processes and random walks, highlighting the potential for multiple NE in certain constructions. The framework provides a practical tool for computing the value and understanding the structure of NE in Markov stopping games with wide-ranging potential applications in economics and finance.

Abstract

In this paper, we propose a new efficient algorithm to compute the value function for zero-sum stopping games featuring two players with opposing interests. This can be seen as a game version of the ''forward algorithm'' for (one-player) optimal stopping problem, first introduced by Irle [6] for discrete-time Markov processes and later revisited by Miclo \& Villeneuve [8] for continuous-time Markov processes on general state spaces. This paper focuses on a game driven by a homogeneous Markov process taking values in a finite state space and also discusses about the number of iterations needed. Illustrated computational implementations for a few particular examples are also provided.
Paper Structure (21 sections, 20 theorems, 124 equations, 8 figures)

This paper contains 21 sections, 20 theorems, 124 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Settings and frequently used results
  3. The algorithm
  4. Forward algorithm for one-player game
  5. First case: $\phi \geq V_0$
  6. Second case: Lg
  7. Auxillary results and discussions
  8. On the choice of the set $S_1 = \{ V_0 > \phi \} \cup \{ \phi = \psi \}$.
  9. A way to create examples that have many NE in the case $\{ V_0 > \phi \} \neq \emptyset$.
  10. Computational aspects.
  11. Examples
  12. Example 1: A birth-death process on $\{0,...,N-1\}$ with reflecting endpoints
  13. Subexample 1.1.
  14. Subexample 1.2.
  15. Subexample 1.3.
  16. ...and 6 more sections

Key Result

Lemma 2.1

Let $B \subset C$ be two subsets of $E$, and let $\sigma$ denote the first jump time of $X$. Suppose that a function $g\in \mathbb{R}^E$ has the property Then,

Figures (8)

  • Figure 1: The evolution of $(V_k)_{k\geq 0}$ (here $V_3 = V$)
  • Figure 2: The evolution of $(V_k)_{k\geq 0}$ (here $V_4=V$)
  • Figure 3: The evolution of $(V_k)_{k\geq 0}$ (here $V_6 = V$)
  • Figure 4: The evolution of stopping regions $(D_k,S_k)_{k= 1,2,3,4}$ from left to right.
  • Figure 5: The functions $\psi, \phi, V_0, V$ on $\{0,1,...,168\}$ ($N = 13$).
  • ...and 3 more figures

Theorems & Definitions (41)

  • Lemma 2.1
  • proof
  • Lemma 2.2
  • proof
  • Remark 2.3
  • Lemma 2.4
  • proof
  • Theorem 2.5
  • proof
  • Theorem 3.1
  • ...and 31 more