Evolving Markov Chains: Unsupervised Mode Discovery and Recognition from Data Streams

TL;DR

This study proposes an online and efficient method to construct Evolving Markov chains (EMCs), which adaptively track transition probabilities, automatically discover modes, and detect mode switches in an online manner and points to the potential of EMCs to efficiently track, model, and understand live, real-world processes.

Abstract

Markov chains are simple yet powerful mathematical structures to model temporally dependent processes. They generally assume stationary data, i.e., fixed transition probabilities between observations/states. However, live, real-world processes, like in the context of activity tracking, biological time series, or industrial monitoring, often switch behavior over time. Such behavior switches can be modeled as transitions between higher-level \emph{modes} (e.g., running, walking, etc.). Yet all modes are usually not previously known, often exhibit vastly differing transition probabilities, and can switch unpredictably. Thus, to track behavior changes of live, real-world processes, this study proposes an online and efficient method to construct Evolving Markov chains (EMCs). EMCs adaptively track transition probabilities, automatically discover modes, and detect mode switches in an online manner. In contrast to previous work, EMCs are of arbitrary order, the proposed update scheme does not rely on tracking windows, only updates the relevant region of the probability tensor, and enjoys geometric convergence of the expected estimates. Our evaluation of synthetic data and real-world applications on human activity recognition, electric motor condition monitoring, and eye-state recognition from electroencephalography (EEG) measurements illustrates the versatility of the approach and points to the potential of EMCs to efficiently track, model, and understand live, real-world processes.

Figures (8)

• Figure 1: (a) The black box process has unknown underlying modes ($M_1$, $M_2$ and $M_3$) and it switches from one mode to another at arbitrary times. In this case, the mode switching sequence is $M_1\to M_2\to M_1\to M_3$ (top figure). The active mode at a given time determines how the symbol sequence is generated (bottom figure). (b) The sequential data generated by the black box process contains neither information about the underlying modes nor the switching points. (c) propAlg processes the sequence in an online manner, and updates the state transition probabilities accordingly to discover the underlying modes. (d) The switching diagram of discovered modes shows which mode is active at each time point. Discovered modes are represented with Markov chains with different transition probabilities, indicated with different colors.
• Figure 2: CAE between true and estimated conditional probabilities for $P_{d}=\mathcal{U}(1500,2000)$. The numbers inside the boxes along the horizontal axis indicate the mode index $i$ of $M_{i} \in \mathcal{M}$.
• Figure 3: Transition plots of true modes (top) and detected modes (bottom) on synthetic data. The heatmap in the bottom plot represents the Hellinger distance, quantifying the detected drift over time.
• Figure 4: Transition plots of true modes (top) and detected modes (bottom) on activity data of subject 9.0. The heatmap in the bottom plot represents the Hellinger distance, quantifying the detected drift over time.
• Figure 5: Result on vibration data under 2.0 HP of load with mode transition sequence of OK → FD:07 → FD:14. The labels of each condition is given along the top horizontal axis. The heatmap in the bottom plot represents the Hellinger distance, quantifying the detected drift over time.

Theorems & Definitions (3)

• Definition 1: Online Mode Discovery and Recognition
• Theorem 1
• proof