Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Representation and De-interleaving of Mixtures of Hidden Markov Processes

Jiadi Bao, Mengtao Zhu, Yunjie Li, Shafei Wang

TL;DR

This work introduces Interleaved Hidden Markov Processes (IHMP) to robustly represent mixtures of hidden Markov sequences observed in interleaved fashion. It develops an exact EM framework and two scalable variational inference methods (MFVI and SVI) to perform posterior inference and parameter learning under noisy and incomplete data, while also deriving a likelihood-ratio based error bound to gauge optimality. The approach reduces the alphabet-partition search space compared to prior IMP models and demonstrates strong de-interleaving performance on synthetic data and real-world tasks such as radar pulse sequence analysis and human motion separation. The results show that SVI often delivers the best accuracy-efficiency trade-off, with strong robustness to missing observations and non-ideal conditions, highlighting practical impact for time-series demixing in complex sensing systems.

Abstract

De-interleaving of the mixtures of Hidden Markov Processes (HMPs) generally depends on its representation model. Existing representation models consider Markov chain mixtures rather than hidden Markov, resulting in the lack of robustness to non-ideal situations such as observation noise or missing observations. Besides, de-interleaving methods utilize a search-based strategy, which is time-consuming. To address these issues, this paper proposes a novel representation model and corresponding de-interleaving methods for the mixtures of HMPs. At first, a generative model for representing the mixtures of HMPs is designed. Subsequently, the de-interleaving process is formulated as a posterior inference for the generative model. Secondly, an exact inference method is developed to maximize the likelihood of the complete data, and two approximate inference methods are developed to maximize the evidence lower bound by creating tractable structures. Then, a theoretical error probability lower bound is derived using the likelihood ratio test, and the algorithms are shown to get reasonably close to the bound. Finally, simulation results demonstrate that the proposed methods are highly effective and robust for non-ideal situations, outperforming baseline methods on simulated and real-life data.

Representation and De-interleaving of Mixtures of Hidden Markov Processes

TL;DR

This work introduces Interleaved Hidden Markov Processes (IHMP) to robustly represent mixtures of hidden Markov sequences observed in interleaved fashion. It develops an exact EM framework and two scalable variational inference methods (MFVI and SVI) to perform posterior inference and parameter learning under noisy and incomplete data, while also deriving a likelihood-ratio based error bound to gauge optimality. The approach reduces the alphabet-partition search space compared to prior IMP models and demonstrates strong de-interleaving performance on synthetic data and real-world tasks such as radar pulse sequence analysis and human motion separation. The results show that SVI often delivers the best accuracy-efficiency trade-off, with strong robustness to missing observations and non-ideal conditions, highlighting practical impact for time-series demixing in complex sensing systems.

Abstract

De-interleaving of the mixtures of Hidden Markov Processes (HMPs) generally depends on its representation model. Existing representation models consider Markov chain mixtures rather than hidden Markov, resulting in the lack of robustness to non-ideal situations such as observation noise or missing observations. Besides, de-interleaving methods utilize a search-based strategy, which is time-consuming. To address these issues, this paper proposes a novel representation model and corresponding de-interleaving methods for the mixtures of HMPs. At first, a generative model for representing the mixtures of HMPs is designed. Subsequently, the de-interleaving process is formulated as a posterior inference for the generative model. Secondly, an exact inference method is developed to maximize the likelihood of the complete data, and two approximate inference methods are developed to maximize the evidence lower bound by creating tractable structures. Then, a theoretical error probability lower bound is derived using the likelihood ratio test, and the algorithms are shown to get reasonably close to the bound. Finally, simulation results demonstrate that the proposed methods are highly effective and robust for non-ideal situations, outperforming baseline methods on simulated and real-life data.
Paper Structure (31 sections, 2 theorems, 40 equations, 9 figures, 1 table)

This paper contains 31 sections, 2 theorems, 40 equations, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Hidden Markov Model with Gaussian Emissions
  4. Interleaved Markov Process
  5. Problem Formulation
  6. The Generative Model
  7. The Combinatorial Problem of IMP and IHMP
  8. Method
  9. Exact Inference
  10. Approximate Inference
  11. Mean-field Variational Inference (MFVI)
  12. Structured Variational Inference (SVI)
  13. Error Analysis
  14. Theoretical Error Lower Bound
  15. Performance Comparison
  16. ...and 16 more sections

Key Result

Theorem 1

Exact inference (E-step) for interleaved mixtures of hidden Markov models is NP-hard Landwehr2008.

Figures (9)

  • Figure 1: Schematic setup for two independent sources.
  • Figure 2: (a) The HMM with multi-variate Gaussian emission. (b) The generative model of IHMP.
  • Figure 3: The mean-field approximation and structured approximation.
  • Figure 4: The error probability of three scenarios was averaged over 100 trials. The results are compared with the theoretical lower bound \ref{['lower bound']} shown as the red line. Scenario 1, scenario 2, scenario 3 corresponds to $[\mu_1^1,\mu_2^1, \mu_1^2, \mu_2^2] \in \{[1,2,3,4],[1,3,2,4],[1,3,4,2]\}$.
  • Figure 5: De-interleaving results of various de-interleaving algorithms.
  • ...and 4 more figures

Theorems & Definitions (2)

  • Theorem 1
  • Theorem 2