Minimum information Markov model

TL;DR

The paper tackles high-dimensional time-series modeling by introducing the minimum information Markov model (MIMM), which separates the stationary distribution from the dependence structure via a KL-inspired divergence-rate framework. By parameterizing the dependence with a linear function $\bm{h}$ and a stationary margin $r$, the authors establish an orthogonal (Pythagorean) geometry that yields a unique MI kernel when it exists and links Gaussian AR/VAR models to this framework. They develop practical estimators—maximum conditional likelihood estimation and Besag's pseudo-likelihood—with efficient inference aided by permutation invariance, plus acceleration strategies like bipartition and online SGD to handle large samples. Simulation studies show tradeoffs among estimators, with MLE being most accurate but often infeasible, and PLE providing scalable performance, complemented by real-data demonstrations on neural recordings (LFP and spike trains) that illustrate model selection and dependence capture. The work offers a principled, flexible approach to modeling high-dimensional time series with potential extensions to broader dependence structures and infinite-state settings.

Abstract

The analysis of high-dimensional time series data has become increasingly important across a wide range of fields. Recently, a method for constructing the minimum information Markov kernel on finite state spaces was established. In this study, we propose a statistical model based on a parametrization of its dependence function, which we call the \textit{Minimum Information Markov Model}. We show that its parametrization induces an orthogonal structure between the stationary distribution and the dependence function, and that the model arises as the optimal solution to a divergence rate minimization problem. In particular, for the Gaussian autoregressive case, we establish the existence of the optimal solution to this minimization problem, a nontrivial result requiring a rigorous proof. For parameter estimation, our approach exploits the conditional independence structure inherent in the model, which is supported by the orthogonality. Specifically, we develop several estimators, including conditional likelihood and pseudo likelihood estimators, for the minimum information Markov model in both univariate and multivariate settings. We demonstrate their practical performance through simulation studies and applications to real-world time series data.

Figures (7)

• Figure 1: Illustration of Proposition \ref{['prop:pythagorean']}.
• Figure 2: Fisher information of the minimum information model equivalent to the Gaussian AR(1) model (Eq. \ref{['eq:fim-ar1']}).
• Figure 3: Decline in the acceptance rate of exchange algorithm during MCLE as the order of AR model increases.
• Figure 4: Estimation time of PLE with synthetic data generated from each AR process across different sample sizes.
• Figure 5: An example of standard scaled LFP; channel = 1.

Theorems & Definitions (38)

• Definition 1: Minimum information Markov kernel (first order)
• Definition 2: $d$-th-order minimum information Markov model
• Definition 3: Divergence rate
• Proposition 1: Generalized Pythagorean Theorem csiszar1987conditional
• proof
• Proposition 2: Fisher information of the first-order minimum information Markov model
• Definition 4
• Example 1: AR(1) model
• Remark 1
• Example 2: AR(2) model