Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Numerical Truncation Approximation with A Posteriori Error Bounds for the Solution of Poisson's Equation

Saied Mahdian, Peter W. Glynn, Yuanyuan Liu

TL;DR

The paper addresses computing the solution to Poisson's equation for forcing functions on large or infinite state spaces by introducing truncation with computable a posteriori error bounds. It first reduces Poisson's equation for continuous-time Markov jump processes to an equivalent discrete-time problem via the embedded chain, enabling Lyapunov-based bounds and a constructive truncation framework. It then derives explicit, computable upper and lower bounds for the solution in both discrete-time and jump-process settings, including procedures to ensure convergence as the truncation set expands. The authors validate the approach with extensive numerical experiments on queueing networks, demonstrating tight error gaps and practical convergence, thereby offering a scalable method with guaranteed accuracy for Poisson-based analyses in large-scale stochastic systems.

Abstract

The solution to Poisson's equation arise in many Markov chain and Markov jump process settings, including that of the central limit theorem, value functions for average reward Markov decision processes, and within the gradient formula for equilibrium Markovian rewards. In this paper, we consider the problem of numerically computing the solution to Poisson's equation when the state space is infinite or very large. In such settings, the state space must be truncated in order to make the problem computationally tractable. In this paper, we provide the first truncation approximation solution to Poisson's equation that comes with provable and computable a posteriori error bounds. Our theory applies to both discrete-time chains and continuous-time jump processes. Through numerical experiments, we show our method can provide highly accurate solutions and tight bounds.

A Numerical Truncation Approximation with A Posteriori Error Bounds for the Solution of Poisson's Equation

TL;DR

The paper addresses computing the solution to Poisson's equation for forcing functions on large or infinite state spaces by introducing truncation with computable a posteriori error bounds. It first reduces Poisson's equation for continuous-time Markov jump processes to an equivalent discrete-time problem via the embedded chain, enabling Lyapunov-based bounds and a constructive truncation framework. It then derives explicit, computable upper and lower bounds for the solution in both discrete-time and jump-process settings, including procedures to ensure convergence as the truncation set expands. The authors validate the approach with extensive numerical experiments on queueing networks, demonstrating tight error gaps and practical convergence, thereby offering a scalable method with guaranteed accuracy for Poisson-based analyses in large-scale stochastic systems.

Abstract

The solution to Poisson's equation arise in many Markov chain and Markov jump process settings, including that of the central limit theorem, value functions for average reward Markov decision processes, and within the gradient formula for equilibrium Markovian rewards. In this paper, we consider the problem of numerically computing the solution to Poisson's equation when the state space is infinite or very large. In such settings, the state space must be truncated in order to make the problem computationally tractable. In this paper, we provide the first truncation approximation solution to Poisson's equation that comes with provable and computable a posteriori error bounds. Our theory applies to both discrete-time chains and continuous-time jump processes. Through numerical experiments, we show our method can provide highly accurate solutions and tight bounds.
Paper Structure (10 sections, 1 theorem, 85 equations, 5 figures)

This paper contains 10 sections, 1 theorem, 85 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Reduction of Poisson's Equation for Markov Jump Processes to the Discrete Time Setting
  3. Bounding Expected Reward Cumulated to Time $\tau(z)$
  4. Numerical Bounds for Poisson's Equation for Discrete-time Markov Chains
  5. Numerical Bounds for the Solution of Poisson's Equation for Markov Jump Processes
  6. Computational Results
  7. A Discrete-time Queue
  8. A Network of Queues 1
  9. A Network of Queues 2
  10. Numerical Results

Key Result

Theorem 1

Suppose that $X$ is an irreducible recurrent Markov chain satisfying A1($q$) for some function $q: S \to \mathbb{R}_+$. Then, for each $x \in A - \{z\}$, provided A is large enough that $\|(I - G)^{-1}\xi \|_{\tilde{K}} < 1$.

Figures (5)

  • Figure 1: Relative error gap for all states in a specific truncation set.
  • Figure 2: Discrete-time queue, value of $upper(\cdot), exact(\cdot), lower(\cdot)$ for each state of the truncation set of Figure \ref{['fig1:figa']}.
  • Figure 3: Relative error gap versus $t$.
  • Figure 4: Network of queues 2, accuracy measures for a specific truncation set.
  • Figure 5: Network of queues 2, approximate relative error gap versus $t$.

Theorems & Definitions (8)

  • Remark 1
  • Remark 2
  • Remark 3
  • Theorem 1
  • Remark 4
  • Remark 5
  • Remark 6
  • Remark 7