Statistical comparison of Hidden Markov Models via Fragment Analysis

TL;DR

This work tackles the computational bottleneck of selecting among competing Hidden Markov Models for long sequences by introducing a fragment-based, likelihood-free framework. It defines a fragment-level metric $\mu_j(r)$ as the probability that a candidate $\mathrm{HMM}_j$ matches the true data-generating model on short sequences of length $r$, enabling unbiased fragment estimators from $k$ samples and a standard $Z$-test to compare $\mu_i(r)$ and $\mu_j(r)$. The approach leverages a forward-factorization representation and Kronecker-product matrices to express $\mu_j(r)$, facilitating efficient computation on short fragments while accommodating non-nested models. A real-data example on ozone concentrations demonstrates that fragment-based tests can decisively distinguish between models with different hidden-state counts, yielding valid $p$-values where full-sequence likelihoods may be unstable or non-nested concerns prevent standard tests. Overall, the method provides scalable, statistically rigorous model comparison for large-scale HMMs with practical guidance on fragment size and asymptotic behavior.

Abstract

Standard practice in Hidden Markov Model (HMM) selection favors the candidate with the highest full-sequence likelihood, although this is equivalent to making a decision based on a single realization. We introduce a \emph{fragment-based} framework that redefines model selection as a formal statistical comparison. For an unknown true model $\mathrm{HMM}_0$ and a candidate $\mathrm{HMM}_j$, let $μ_j(r)$ denote the probability that $\mathrm{HMM}_j$ and $\mathrm{HMM}_0$ generate the same sequence of length~$r$. We show that if $\mathrm{HMM}_i$ is closer to $\mathrm{HMM}_0$ than $\mathrm{HMM}_j$, there exists a threshold $r^{*}$ -- often small -- such that $μ_i(r)>μ_j(r)$ for all $r\geq r^{*}$. Sampling $k$ independent fragments yields unbiased estimators $\hatμ_j(r)$ whose differences are asymptotically normal, enabling a straightforward $Z$-test for the hypothesis $H_0\!:\,μ_i(r)=μ_j(r)$. By evaluating only short subsequences, the procedure circumvents full-sequence likelihood computation and provides valid $p$-values for model comparison.

Figures (1)

• Figure 1: Plot of the data in Table \ref{['tab:rates']}. No overlap observed in the studied interval.