Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reduction and classification of higher-order Markov chains for categorical data

Christophe Gallesco, Caio Teodore Genovese Huss Oliveira, Daniel Yasumasa Takahashi

TL;DR

The paper addresses the challenge of analyzing high-memory ($m$-th order) Markov chains on a finite alphabet by introducing a skeleton framework that captures minimal prohibitive contexts. It proves that the recurrent-class structure and periods of the original chain are fully determined by a binary skeleton matrix $\mathbb{M}$ associated with skeleton $\mathcal{S}$, and it provides an explicit, pruning-based algorithm to extract $\mathcal{S}$ from the transition kernel. The framework yields practical irreducibility criteria and can substantially reduce computation, as demonstrated by a $10$-th order example where the skeleton order $K$ is much smaller than $m$. Overall, the method enables scalable, intrinsic analysis of higher-order Markov chains, with concrete gains in efficiency for identifying recurrent classes and irreducibility properties.

Abstract

Categorical time series models are powerful tools for understanding natural phenomena. Most available models can be formulated as special cases of $m$-th order Markov chains, for $m\geq 1$. Despite their broad applicability, theoretical research has largely focused on first-order Markov chains, mainly because many properties of higher-order chains can be analyzed by reducing them to first-order chains on an enlarged alphabet. However, the resulting first-order representation is sparse and possesses a highly structured transition kernel, a feature that has not been fully exploited. In this work, we study finite-alphabet Markov chains with arbitrary memory length and introduce a new reduction framework for their structural classification. We define the skeleton of a transition kernel, an object that captures the intrinsic pattern of transition probability constraints in a higher-order Markov chain. We show that the class structure of a binary matrix associated with the skeleton completely determines the recurrent classes and their periods in the original chain. We also provide an explicit algorithm for efficiently extracting the skeleton, which in many cases yields substantial computational savings. Applications include simple criteria for irreducibility and essential irreducibility of higher-order Markov chains and a concrete illustration based on a 10th-order Markov chain.

Reduction and classification of higher-order Markov chains for categorical data

TL;DR

The paper addresses the challenge of analyzing high-memory (-th order) Markov chains on a finite alphabet by introducing a skeleton framework that captures minimal prohibitive contexts. It proves that the recurrent-class structure and periods of the original chain are fully determined by a binary skeleton matrix associated with skeleton , and it provides an explicit, pruning-based algorithm to extract from the transition kernel. The framework yields practical irreducibility criteria and can substantially reduce computation, as demonstrated by a -th order example where the skeleton order is much smaller than . Overall, the method enables scalable, intrinsic analysis of higher-order Markov chains, with concrete gains in efficiency for identifying recurrent classes and irreducibility properties.

Abstract

Categorical time series models are powerful tools for understanding natural phenomena. Most available models can be formulated as special cases of -th order Markov chains, for . Despite their broad applicability, theoretical research has largely focused on first-order Markov chains, mainly because many properties of higher-order chains can be analyzed by reducing them to first-order chains on an enlarged alphabet. However, the resulting first-order representation is sparse and possesses a highly structured transition kernel, a feature that has not been fully exploited. In this work, we study finite-alphabet Markov chains with arbitrary memory length and introduce a new reduction framework for their structural classification. We define the skeleton of a transition kernel, an object that captures the intrinsic pattern of transition probability constraints in a higher-order Markov chain. We show that the class structure of a binary matrix associated with the skeleton completely determines the recurrent classes and their periods in the original chain. We also provide an explicit algorithm for efficiently extracting the skeleton, which in many cases yields substantial computational savings. Applications include simple criteria for irreducibility and essential irreducibility of higher-order Markov chains and a concrete illustration based on a 10th-order Markov chain.
Paper Structure (5 sections, 6 theorems, 10 equations, 5 figures)

This paper contains 5 sections, 6 theorems, 10 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Theoretical results
  3. Skeleton determination algorithm
  4. Example of application
  5. Computational cost

Key Result

Theorem 1

Consider a Markov chain of order $m$, $X$, with skeleton of order $K$. If $\mathbb{M}$ has $N \geq 1$ closed classes $\mathcal{C}_1,\dots,\mathcal{C}_N$, then $\mathbb{P}$ has $N$ recurrent classes $\mathcal{R}_1,\dots,\mathcal{R}_N$. The classes $\mathcal{R}_i$ satisfy for $i\in\{1,\dots,N\}$. Moreover, the period of $\mathcal{C}_i$ is equal to the period of $\mathcal{R}_i$ for $i\in\{1,\dots,N\

Figures (5)

  • Figure 1: Part of the tree $T$ corresponding to the children of node $w$. On the feft-hand figure, for each node we show the corresponding transition probabilities $[\hat{p}(\cdot,0),\hat{p}(\cdot,1)]$. On the right-hand figure, we show the transition vectors $[t_0,t_1]$ associated to each node. The node $00w$ (in red) will not be cut, while nodes $01w$ and $11w$ (in blue) will be cut off the tree.
  • Figure 2: Resulting tree after step 1.
  • Figure 3: Tree $\tau$ of the 10-th order Markov chain $X$. The circled nodes correspond to prohibited transitions toward $1$. All the other contexts gives positive probabilities toward 0 and 1.
  • Figure 4: Skeleton $\mathcal{S}$ of $\mathbf{X}$, contexts that prohibit transitions are circled.
  • Figure 5: Skeleton matrix $\mathbb{M}$ of $\mathcal{S}$: 1's in red and 0's in grey.

Theorems & Definitions (10)

  • Definition 1
  • Definition 2
  • Theorem 1
  • Definition 3
  • Corollary 1
  • Proposition 1
  • Proposition 2
  • Proposition 3
  • proof
  • Proposition 4