Computable Bounds on the Solution to Poisson's Equation for General Harris Chains
Peter W. Glynn, Na Lin, Yuanyuan Liu
TL;DR
The paper advances Poisson's equation analysis for general Harris chains by constructing a direct regenerative representation of the solution g^* via a randomized stopping time tied to a small set and Lyapunov drift conditions, yielding computable bounds. It proves a computable uniform bound on E_x f(X_n) and provides a potential kernel representation g^*(x)=lim_{n->infty} sum_{i=0}^{n-1} E_x f_c(X_i) + c under mild conditions. The work extends prior results to general Harris chains with m>=1, offering tighter, computable bounds than existing m=1 analyses and presenting a GI/G/1 waiting-time example that demonstrates improved asymptotics over previous bounds. These results support martingale constructions, bias analysis in steady-state simulation, and perturbation studies in Markov processes and average-reward control.
Abstract
Poisson's equation is fundamental to the study of Markov chains, and arises in connection with martingale representations and central limit theorems for additive functionals, perturbation theory for stationary distributions, and average reward Markov decision process problems. In this paper, we develop a new probabilistic representation for the solution of Poisson's equation, and use Lyapunov functions to bound this solution representation explicitly. In contrast to most prior work on this problem, our bounds are computable. Our contribution is closely connected to recent work of Herve and Ledoux (2025), in which they focus their study on a special class of Harris chains satisfying a particular small set condition. However, our theory covers general Harris chains, and often provides a tighter bound. In addition to the new bound and representation, we also develop a computable uniform bound on marginal expectations for Harris chains, and a computable bound on the potential kernel representation of the solution to Poisson's equation.