Time Series Clustering with General State Space Models via Stochastic Variational Inference

TL;DR

The proposed method is the first computationally feasible one for time series clustering based on general (possibly nonlinear, non-Gaussian) state space models and is effective for clustering, parameter estimation, and estimating the number of clusters.

Abstract

In this paper, we propose a novel method of model-based time series clustering with mixtures of general state space models (MSSMs). Each component of MSSMs is associated with each cluster. An advantage of the proposed method is that it enables the use of time series models appropriate to the specific time series. This not only improves clustering and prediction accuracy but also enhances the interpretability of the estimated parameters. The parameters of the MSSMs are estimated using stochastic variational inference, a subtype of variational inference. The proposed method estimates the latent variables of an arbitrary state space model by using neural networks with a normalizing flow as a variational estimator. The number of clusters can be estimated using the Bayesian information criterion. In addition, to prevent MSSMs from converging to the local optimum, we propose several optimization tricks, including an additional penalty term called entropy annealing. To our best knowledge, the proposed method is the first computationally feasible one for time series clustering based on general (possibly nonlinear, non-Gaussian) state space models. Experiments on simulated datasets show that the proposed method is effective for clustering, parameter estimation, and estimating the number of clusters.

Figures (6)

• Figure 1: Overview of proposed method.
• Figure 2: Actual time series
• Figure 3: Estimated latent variables
• Figure 5: Actual time series
• Figure 6: Estimated latent variables