Quick Adaptive Ternary Segmentation: An Efficient Decoding Procedure For Hidden Markov Models

TL;DR

This work tackles decoding hidden state sequences in large-scale HMMs where the observation sequence is long relative to the number of states. It introduces Quick Adaptive Ternary Segmentation (QATS), a greedy divide-and-conquer decoder that uses ternary segmentation and optimistic search to achieve polylogarithmic runtime in the sequence length $n$ and cubic complexity in the number of states $m$. The method precomputes log-likelihood components, applies local two- and three-segment maximizations via subroutines OSH2 and OSH3, and yields an admissible QATS-path with substantial speedups over Viterbi and PMAP in simulations, while maintaining competitive accuracy; a reference implementation is available in the R-package QATS. Overall, QATS enables scalable decoding for large HMMs, offering practical gains in speed for applications with infrequent state changes and opening avenues for parallelization and extensions to broader change-point detection problems.

Abstract

Hidden Markov models (HMMs) are characterized by an unobservable Markov chain and an observable process -- a noisy version of the hidden chain. Decoding the original signal from the noisy observations is one of the main goals in nearly all HMM based data analyses. Existing decoding algorithms such as Viterbi and the pointwise maximum a posteriori (PMAP) algorithm have computational complexity at best linear in the length of the observed sequence, and sub-quadratic in the size of the state space of the hidden chain. We present Quick Adaptive Ternary Segmentation (QATS), a divide-and-conquer procedure with computational complexity polylogarithmic in the length of the sequence, and cubic in the size of the state space, hence particularly suited for large scale HMMs with relatively few states. It also suggests an effective way of data storage as specific cumulative sums. In essence, the estimated sequence of states sequentially maximizes local likelihood scores among all local paths with at most three segments, and is meanwhile admissible. The maximization is performed only approximately using an adaptive search procedure. Our simulations demonstrate the speedups offered by QATS in comparison to Viterbi and PMAP, along with a precision analysis. An implementation of QATS is in the R-package QATS on GitHub.

Figures (10)

• Figure 1: Some stages of QATS. Black line segments represent the true, original path $\boldsymbol{x}^o$, and points correspond to $\boldsymbol{y}$. Left: After a few iterations of QATS, the best local path on $S_1$ with at most three segments is constant equal to $\widehat{z}_1 = 1$ (white segment with gray border). The best local path on $S_2$ with at most three segments and previous states $\widehat{z}_{1}=1$ is constant equal to $\widehat{z}_{2}= 3$. The next interval to investigate is $S_3$. Middle: In step (1), the search for the best local path on $S_3$ with at most three segments yields a path with $\widehat{c}=3$ intervals (see \ref{['Fig:Gain_Functions']}). This replaces the old interval $S_3$ by new ones: $S_3=S^{1}$, $S_4=S^{2}$, and $S_5=S^{3}$; and the scalar $\widehat{z}_3=1$ by the vector of states: $\widehat{z}_{3:5}=(1,3,2)$. Step (2) sets the new $S_3$ as the next interval to investigate. Right: Step (1) replaced the old $S_3$ by $S_3=S^1$ and $S_4=S^2$, and the old $\widehat{z}_3$ by $\widehat{z}_{3:4}=(2,1)$, whereas step (2) sets the new $S_3$ as the current interval. Applying steps (1--2) to $S_3$, followed by $S_4$ and $S_5$, will not yield any changes. The next interval to investigate will be $S_6$.
• Figure 2: Plots of ${\cal H}^2 = \log H^2$ (left) and ${\cal H}^3 = \log H^3$ (right) in the setting of the left plot of \ref{['Fig:Ternary_Segmentation']}, when the interval $S_3$ is being investigated. Dashed lines (black, or white on black background) show the true change points of the hidden chain $\boldsymbol{x}^o$, whereas the solid black line (left) and point (right) correspond to the respective global maxima of each map. The gray line (left) and triangle (right) correspond to the output of Algorithms \ref{['Alg:OSH2']} and \ref{['Alg:OSH3']}, respectively.
• Figure 3: Plots of ${\cal H}^2 = \log H^2$ (left) and ${\cal H}^3 = \log H^3$ (right). Local maxima (solid gray lines or gray points with black border) are located either on the boundaries, on the diagonal (for ${\cal H}^3$), on true change points or pairs thereof (dashed lines).
• Figure 4: Median time ratios against exit probability $p$ using $\log$-scales.
• Figure 5: Median quantile time ratio (left) and times (right) on $\log$-scales, with $10\%$- and $90$%-quantile curves, for $\sigma = 1.0$ and $n = 10^6+1$.

Theorems & Definitions (26)

• Lemma 2.1: From Kovacs_2020
• Lemma 2.2
• Lemma 2.3
• Corollary 2.4
• Lemma 2.5
• proof
• Theorem 2.6
• Example 3.1
• Theorem 3.2
• Lemma 3.3