Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Regeneration-based a Posteriori Error Bound for a Markov Chain Stationary Distribution Truncation Algorithm

Peter W. Glynn, Zeyu Zheng

TL;DR

The paper addresses accurate truncation of infinite or large state-space Markov chains by introducing a regeneration-based, a posteriori error bound that leverages a Lyapunov function. It derives a regenerative representation of the stationary distribution $\pi$ and constructs computable upper and lower bounds on the associated expectations, yielding explicit bounds on $|\pi r-\tilde{\pi} r|$ and the weighted total variation $\|\pi-\tilde{\pi}\|_r$ without solving linear programs. The key contributions include Theorem 1, which decomposes regenerative cycles into excursions to provide a computable bound, and Theorem 2, which gives a practical total-variation bound; together they enable tight, data-driven guarantees for truncation accuracy. Empirical results on a G/M/1 model and a random walk demonstrate comparable performance to LP-based bounds while delivering tight a posteriori guarantees and avoiding LPs, enhancing practical computability for large-scale truncation problems.

Abstract

When the state space of a discrete state space positive recurrent Markov chain is infinite or very large, it becomes necessary to truncate the state space in order to facilitate numerical computation of the stationary distribution. This paper develops a new approach for bounding the truncation error that arises when computing approximations to the stationary distribution. This rigorous a posteriori error bound exploits the regenerative structure of the chain and assumes knowledge of a Lyapunov function. Because the bound is a posteriori (and leverages the computations done to calculate the stationary distribution itself), it tends to be much tighter than a priori bounds. The bound decomposes the regenerative cycle into a random number of excursions from a set $K$ defined in terms of the Lyapunov function into the complement of the truncation set $A$. The bound can be easily computed, and does not (for example) involve a linear program, as do some other error bounds.

A Regeneration-based a Posteriori Error Bound for a Markov Chain Stationary Distribution Truncation Algorithm

TL;DR

The paper addresses accurate truncation of infinite or large state-space Markov chains by introducing a regeneration-based, a posteriori error bound that leverages a Lyapunov function. It derives a regenerative representation of the stationary distribution and constructs computable upper and lower bounds on the associated expectations, yielding explicit bounds on and the weighted total variation without solving linear programs. The key contributions include Theorem 1, which decomposes regenerative cycles into excursions to provide a computable bound, and Theorem 2, which gives a practical total-variation bound; together they enable tight, data-driven guarantees for truncation accuracy. Empirical results on a G/M/1 model and a random walk demonstrate comparable performance to LP-based bounds while delivering tight a posteriori guarantees and avoiding LPs, enhancing practical computability for large-scale truncation problems.

Abstract

When the state space of a discrete state space positive recurrent Markov chain is infinite or very large, it becomes necessary to truncate the state space in order to facilitate numerical computation of the stationary distribution. This paper develops a new approach for bounding the truncation error that arises when computing approximations to the stationary distribution. This rigorous a posteriori error bound exploits the regenerative structure of the chain and assumes knowledge of a Lyapunov function. Because the bound is a posteriori (and leverages the computations done to calculate the stationary distribution itself), it tends to be much tighter than a priori bounds. The bound decomposes the regenerative cycle into a random number of excursions from a set defined in terms of the Lyapunov function into the complement of the truncation set . The bound can be easily computed, and does not (for example) involve a linear program, as do some other error bounds.
Paper Structure (3 sections, 3 theorems, 53 equations, 1 table)

This paper contains 3 sections, 3 theorems, 53 equations, 1 table.

Table of Contents

  1. Introduction
  2. The Regenerative Bounds
  3. Numerical experiment

Key Result

Proposition 1

Suppose that $X$ is irreducible and positive recurrent. Then, for each $x\in S$, $\tilde{\pi}_n(x)\rightarrow \pi(x)$ as $n\rightarrow\infty$.

Theorems & Definitions (6)

  • Proposition 1
  • Remark 1
  • Remark 2
  • Theorem 1
  • Theorem 2
  • Remark 3