A note on the state occupancy distribution for Markov chains

TL;DR

This work extends Shah's explicit two-state occupancy distribution to countable-state discrete-time Markov chains by formulating the problem in terms of the occupancy count $N_n$ and its conditional probabilities $g_i(n,k)$. It develops a matrix-block decomposition of the transition matrix and a generating-function framework, yielding a compact expression $G(t,k)=[(I-Bt)^{-1}A]^k (I-Bt)^{-1}\boldsymbol{1}$ that underpins both analytic results and computational approaches. In the two-state case, the paper provides exact formulas: a binomial form when $p+q=1$, and detailed sum-based expressions for $g_0(n,k)$ and $g_1(n,k)$ when $p+q\neq 1$, including boundary values $g_0(n,n)=(1-p)^n$ and $g_1(n,0)=(1-q)^n$. The discussion links these results to first-passage times and potential theory, notes practical limitations due to dependence on powers of submatrices, and outlines potential extensions to moment calculations and total-cost processes via auxiliary generating functions.

Abstract

In a recent paper, Shah [arXiv:2502.03073] derived an explicit expression for the distribution of occupancy times for a two-state Markov chain, using a method based on enumerating sample paths. We consider here the more general problem of finding the distribution of occupancy times for countable-state Markov chains in discrete time. Our approach, which employs generating functions, leads to arguably simpler formulae for the occupancy distribution for the two-state chain.