HOMC: A MATLAB Package for Higher Order Markov Chains
Jianhong Xu
TL;DR
This work addresses the lack of dedicated tools for higher-order Markov chains by presenting HOMC, a MATLAB package for $m$-th order chains. It delivers a comprehensive tensor-algebra toolkit centered on the box product for $m$-th order, $n$-dimensional tensors, enabling computation of the $k$-step transition tensor, the limiting distribution, the ever-reaching probability tensor, and the mean first passage time tensor. The package also provides diagnostics for ergodicity/regularity, methods to form the reduced first-order chain, and state classification via the ever-reaching tensor, with both direct and iterative approaches for MFPT. These capabilities support numerical experimentation and rapid prototyping in the analysis and application of higher-order Markov chains.
Abstract
We present a MATLAB package, which is the first of its kind, for Higher Order Markov Chains (HOMC). It can be used to easily compute all important quantities in our recent works relevant to higher order Markov chains, such as the $k$-step transition tensor, limiting probability distribution, ever-reaching probability tensor, and mean first passage time tensor. It can also be used to check whether a higher order chain is ergodic or regular, to construct the transition matrix of the associated reduced first order chain, and to determine whether a state is recurrent or transient. A key function in the package is an implementation of the tensor ``box'' product which has a probabilistic interpretation and is different from other tensor products in the literature. This HOMC package is useful to researchers and practitioners alike for tasks such as numerical experimentation and algorithm prototyping involving higher order Markov chains.