Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Critical Account of Perturbation Analysis of Markov Chains

Karim Abbas, Joost Berkhout, Bernd Heidergott

Abstract

Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite and denumerable Markov chains and introduces new bounds based on series expansions. We discuss a series of examples to illustrate the applicability and numerical efficiency of the various bounds. Specifically, we address the question on how the bounds developed for finite Markov chains behave as the size of the system grows. In addition, we provide for the first time an analysis of the relative error of these bounds. For the case of a scaled perturbation we show that perturbation bounds can be used to analyze stability of a stable Markov chain with respect to perturbation with an unstable chain.

A Critical Account of Perturbation Analysis of Markov Chains

Abstract

Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite and denumerable Markov chains and introduces new bounds based on series expansions. We discuss a series of examples to illustrate the applicability and numerical efficiency of the various bounds. Specifically, we address the question on how the bounds developed for finite Markov chains behave as the size of the system grows. In addition, we provide for the first time an analysis of the relative error of these bounds. For the case of a scaled perturbation we show that perturbation bounds can be used to analyze stability of a stable Markov chain with respect to perturbation with an unstable chain.

Paper Structure

This paper contains 14 sections, 7 theorems, 147 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Perturbation Analysis
  3. Preliminaries and Basic Definitions
  4. Condition Number Perturbation Bounds for Finite Chains
  5. The Choice of Norms in Perturbation Analysis
  6. Perturbation Bounds
  7. Explicit Perturbation Bounds for the Two-State Chain (Finite State Space)
  8. An Elaborate Perturbation Analysis of a Queueing System
  9. The Basic Model
  10. Discussion of Literature
  11. The Infinite Capacity M/G/1 Queue with Breakdowns (Denumerable State Space)
  12. The M/G/1/N Queue with Breakdowns (Finite State Space)
  13. Discussion of Results
  14. Conclusion

Key Result

Proposition 1

Let $P$ be irreducible. If for at least one $i \in S$ it holds that $|| _i \! P || < 1$, then $P$ is positive recurrent.

Figures (4)

  • Figure 1: Perturbation bounds for $\|\pi_{P(\theta)}^\top-\pi_P^\top\|_\infty$ with $\theta \in (0,1]$, where $P(\theta)=(1-\theta)P+\theta R$ for randomly generated $P$ and $R$ consisting of $40$ states.
  • Figure 2: Relative errors of the perturbation bounds for $\|\pi_{P(\theta)}^\top -\pi_P^\top \|_\infty$ with $\theta \in (0,1]$, where $P(\theta)=(1-\theta)P+\theta R$ for randomly generated $P$ and $R$ consisting of $40$ states.
  • Figure 3: The true change in probability of more than 2 customers in the system vs. the strong stability bound.
  • Figure 4: The relative absolute error for approximating the $| \pi_\theta^\top f - \pi_0^\top f |$ with SEB($K$) with $K=1,2$ and $3$.

Theorems & Definitions (20)

  • Example 1
  • Proposition 1
  • Example 2
  • Lemma 1
  • Example 3
  • Lemma 2
  • Remark 1
  • Theorem 1
  • Remark 2
  • Lemma 3
  • ...and 10 more