Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantum Dimension Reduction of Hidden Markov Models

Rishi Sundar, Thomas Elliott

TL;DR

This work tackles the memory cost of classical hidden Markov models (HMMs) and their quantum counterparts by introducing a dilation-based pipeline that transforms any finite, stationary, ergodic HMM into a deterministic process on an augmented output alphabet. The resulting q-sample is representable as a normal infinite matrix product state (iMPS), enabling stable variational truncation to a reduced bond dimension $\tilde{d}$ and subsequent reconstruction of a compressed quantum sampler for the original outputs. The authors demonstrate the approach on a tunable toy non-deterministic source and a speech-derived HMM, showing favorable memory–distortion trade-offs compared to classical reductions and highlighting how the entanglement structure induced by dilation governs compressibility. They also reveal that the labeling function used in dilation materially affects compression performance, suggesting a design parameter for optimizing quantum dimension reduction with potential practical quantum-sampler implementations.

Abstract

Hidden Markov models (HMMs) are ubiquitous in time-series modelling, with applications ranging from chemical reaction modelling to speech recognition. These HMMs are often large, with high-dimensional memories. A recently-proposed application of quantum technologies is to execute quantum analogues of HMMs. Such quantum HMMs (QHMMs) are strictly more expressive than their classical counterparts, enabling the construction of more parsimonious models of stochastic processes. However, state-of-the-art techniques for QHMM compression, based on tensor networks, are only applicable for a restricted subset of HMMs, where the transitions are deterministic. In this work we introduce a pipeline by which \emph{any} finite, ergodic HMM can be compressed in this manner, providing a route for effective quantum dimension reduction of general HMMs. We demonstrate the method on both a simple toy model, and on a speech-derived HMM trained from data, obtaining favourable memory--accuracy trade-offs compared to classical compression approaches.

Quantum Dimension Reduction of Hidden Markov Models

TL;DR

This work tackles the memory cost of classical hidden Markov models (HMMs) and their quantum counterparts by introducing a dilation-based pipeline that transforms any finite, stationary, ergodic HMM into a deterministic process on an augmented output alphabet. The resulting q-sample is representable as a normal infinite matrix product state (iMPS), enabling stable variational truncation to a reduced bond dimension and subsequent reconstruction of a compressed quantum sampler for the original outputs. The authors demonstrate the approach on a tunable toy non-deterministic source and a speech-derived HMM, showing favorable memory–distortion trade-offs compared to classical reductions and highlighting how the entanglement structure induced by dilation governs compressibility. They also reveal that the labeling function used in dilation materially affects compression performance, suggesting a design parameter for optimizing quantum dimension reduction with potential practical quantum-sampler implementations.

Abstract

Hidden Markov models (HMMs) are ubiquitous in time-series modelling, with applications ranging from chemical reaction modelling to speech recognition. These HMMs are often large, with high-dimensional memories. A recently-proposed application of quantum technologies is to execute quantum analogues of HMMs. Such quantum HMMs (QHMMs) are strictly more expressive than their classical counterparts, enabling the construction of more parsimonious models of stochastic processes. However, state-of-the-art techniques for QHMM compression, based on tensor networks, are only applicable for a restricted subset of HMMs, where the transitions are deterministic. In this work we introduce a pipeline by which \emph{any} finite, ergodic HMM can be compressed in this manner, providing a route for effective quantum dimension reduction of general HMMs. We demonstrate the method on both a simple toy model, and on a speech-derived HMM trained from data, obtaining favourable memory--accuracy trade-offs compared to classical compression approaches.
Paper Structure (25 sections, 8 theorems, 63 equations, 6 figures)

This paper contains 25 sections, 8 theorems, 63 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Classical models of stochastic processes
  4. Quantum models of stochastic processes
  5. Matrix product states for deterministic models
  6. Difficulties with non-deterministic HMMs
  7. Theoretical Results
  8. Dilation to a deterministic process
  9. Constructing the $q$–sample iMPS
  10. Variational truncation at reduced bond dimension
  11. Reconstructing an effective model
  12. Quantifying distance from exact process
  13. Analytic bounds via fidelity divergence rates
  14. Examples and Analysis
  15. Illustrative model: $N$-state simple non-deterministic source
  16. ...and 10 more sections

Key Result

Lemma 1

For each state $s\in\mathcal{S}$ and composite symbol $(x,y)\in\mathcal{X}\times\mathcal{Y}$ there is at most one successor $s'\in\mathcal{S}$ with $T^{(x,y)}_{s's} > 0$.

Figures (6)

  • Figure 1: Tensor network representations of stochastic models and their compression. (a) A transition matrix $T^x_{ij}$ of a HMM can be represented as a rank-3 tensor; analogously, (b) the transition matrix of the dilated process $T^{(x,y)}_{ij}$ can be represented by a rank-4 tensor, as can (c) its element-wise square root $A^{(x,y)}_{ij}$. An array of these form an injective iMPS when the two output legs are grouped, enabling the application of iMPS compression methods to deduce a dimension-reduced iMPS with site tensors $\tilde{A}_{ij}^{(x,y)}$. The (d) transfer matrix of the reduced iMPS defines a quantum channel $\tilde{\mathcal{E}}$ when a Kronecker-$\delta$ is applied to the two visible output legs, implementing a compressed model of the original process. This can then (e) be paired with the original process to efficiently calculate their CDR.
  • Figure 2: HMM representation of the generalised $N$-state tunable non-deterministic source (TNS). Transition labels show 'probability $\mid$ emitted symbol', with the auxiliary label symbol in red.
  • Figure 3: TNS model compression results: co-emission divergence rate $R_C$ between the original process and the reconstructed effective model as a function of truncated bond dimension $\tilde{d}$. Each curve corresponds to a different choice of the number of states $N$ and internal parameter $p$.
  • Figure 4: Schmidt spectra of the dilated iMPS of the TNS for fixed $N = 15$ and varied $p$, ordered by decreasing magnitude. The shape of the specra can vary, but parameter regimes with faster-decaying spectra are expected to be more amenable to truncation to small bond dimension.
  • Figure 5: Speech-derived HMM compression results: comparison of quantum dimension reduction and a classical greedy state-merging baseline. Horizontal axis reports memory dimension as classical state count or quantum bond dimension. Vertical axis reports CDR $R_C$ (bits per symbol, log scale). Values below $10^{-9}$ are displayed at $10^{-9}$ for visibility.
  • ...and 1 more figures

Theorems & Definitions (16)

  • Lemma 1: Deterministicity of the dilation
  • proof
  • Lemma 2: Preservation of observable statistics
  • proof
  • Lemma 3: Ergodicity of the dilation
  • proof
  • Lemma 4: Square-root tensors reproduce word statistics for unifilar generators
  • proof
  • Lemma 5: Classical fidelity rate bounded by QFDR
  • proof
  • ...and 6 more