Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LDTC: Lifelong deep temporal clustering for multivariate time series

Zhi Wang, Yanni Li, Pingping Zheng, Yiyuan Jiao

TL;DR

LDTC addresses the challenge of clustering evolving, unlabeled multivariate time series by integrating dimensionality reduction and temporal clustering into an end-to-end unsupervised framework. It introduces a Convolution Temporal AutoEncoder (CTAE) with two dilated causal CNN blocks and Attention+BiLSTM, coupled with a Temporal Clustering Layer and a hierarchical two-objective optimization to learn high-quality latent representations $Z$ and discriminable cluster centroids $oldsymbol{bc}_j$. A lifelong learning mechanism is implemented via a Model Pool that enables fully dynamic model expansion and mixture replay, allowing LDTC to learn new tasks without catastrophic forgetting. Empirical results on seven real-world datasets demonstrate superior clustering performance over baselines and reveal effective continual adaptation under non-stationary data conditions. The work advances unsupervised lifelong learning for temporal clustering and offers a scalable approach for real-world streaming multivariate time series analysis.

Abstract

Clustering temporal and dynamically changing multivariate time series from real-world fields, called temporal clustering for short, has been a major challenge due to inherent complexities. Although several deep temporal clustering algorithms have demonstrated a strong advantage over traditional methods in terms of model learning and clustering results, the accuracy of the few algorithms are not satisfactory. None of the existing algorithms can continuously learn new tasks and deal with the dynamic data effectively and efficiently in the sequential tasks learning. To bridge the gap and tackle these issues, this paper proposes a novel algorithm \textbf{L}ifelong \textbf{D}eep \textbf{T}emporal \textbf{C}lustering (\textbf{LDTC}), which effectively integrates dimensionality reduction and temporal clustering into an end-to-end deep unsupervised learning framework. Using a specifically designed autoencoder and jointly optimizing for both the latent representation and clustering objective, the LDTC can achieve high-quality clustering results. Moreover, unlike any previous work, the LDTC is uniquely equipped with the fully dynamic model expansion and rehearsal-based techniques to effectively learn new tasks and to tackle the dynamic data in the sequential tasks learning without the catastrophic forgetting or degradation of the model accuracy. Experiments on seven real-world multivariate time series datasets show that the LDTC is a promising method for dealing with temporal clustering issues effectively and efficiently.

LDTC: Lifelong deep temporal clustering for multivariate time series

TL;DR

LDTC addresses the challenge of clustering evolving, unlabeled multivariate time series by integrating dimensionality reduction and temporal clustering into an end-to-end unsupervised framework. It introduces a Convolution Temporal AutoEncoder (CTAE) with two dilated causal CNN blocks and Attention+BiLSTM, coupled with a Temporal Clustering Layer and a hierarchical two-objective optimization to learn high-quality latent representations and discriminable cluster centroids . A lifelong learning mechanism is implemented via a Model Pool that enables fully dynamic model expansion and mixture replay, allowing LDTC to learn new tasks without catastrophic forgetting. Empirical results on seven real-world datasets demonstrate superior clustering performance over baselines and reveal effective continual adaptation under non-stationary data conditions. The work advances unsupervised lifelong learning for temporal clustering and offers a scalable approach for real-world streaming multivariate time series analysis.

Abstract

Clustering temporal and dynamically changing multivariate time series from real-world fields, called temporal clustering for short, has been a major challenge due to inherent complexities. Although several deep temporal clustering algorithms have demonstrated a strong advantage over traditional methods in terms of model learning and clustering results, the accuracy of the few algorithms are not satisfactory. None of the existing algorithms can continuously learn new tasks and deal with the dynamic data effectively and efficiently in the sequential tasks learning. To bridge the gap and tackle these issues, this paper proposes a novel algorithm \textbf{L}ifelong \textbf{D}eep \textbf{T}emporal \textbf{C}lustering (\textbf{LDTC}), which effectively integrates dimensionality reduction and temporal clustering into an end-to-end deep unsupervised learning framework. Using a specifically designed autoencoder and jointly optimizing for both the latent representation and clustering objective, the LDTC can achieve high-quality clustering results. Moreover, unlike any previous work, the LDTC is uniquely equipped with the fully dynamic model expansion and rehearsal-based techniques to effectively learn new tasks and to tackle the dynamic data in the sequential tasks learning without the catastrophic forgetting or degradation of the model accuracy. Experiments on seven real-world multivariate time series datasets show that the LDTC is a promising method for dealing with temporal clustering issues effectively and efficiently.
Paper Structure (18 sections, 8 equations, 2 figures, 2 tables)

This paper contains 18 sections, 8 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Temporal clustering problem.
  4. Traditional temporal clustering algorithms.
  5. Deep temporal clustering algorithms.
  6. A Novel Lifelong Deep Temporal Clustering Solution LDTC
  7. An overview of LDTC
  8. A novel temporal clustering neural network.
  9. The main components for the effective latent representation learning in LDTC.
  10. Model training and task referring
  11. Lifelong temporal clustering learning of the LDTC
  12. Fully dynamic model expansion.
  13. Resistance to forgetting and degradation via the mixture replay.
  14. Experiments
  15. Datasets and evaluated metrics
  16. ...and 3 more sections

Figures (2)

  • Figure 1: The framework of the proposed LDTC. (a) The two stages of Pre-training phase and Training phase of LDTC. (b) The schematic diagram of the lifelong learning mechanisms embedded in the LDTC, where the lifelong temporal clustering flow of the LDTC details in Sec. 3.3.
  • Figure 2: The algorithm performance for learning new tasks. (a) The performance on dataset ArabicDigits. (b) The performance on dataset Uwave. (c) The training time of models: LDTC, DTC and USRL, respectively.