Concrete Dense Network for Long-Sequence Time Series Clustering

TL;DR

This paper tackles long-sequence time series clustering by introducing LoSTer, a dense two-view autoencoder that optimizes the canonical k-means objective end-to-end via the Gumbel-softmax. By avoiding autoregressive decoders and soft-k-means surrogates, LoSTer achieves state-of-the-art clustering accuracy and much faster training on long sequences, validated across 17 UCR datasets and two large real-world retail corpora. A dual contrastive learning scheme and a carefully designed loss function (reconstruction, differentiable k-means, instance and cluster contrasts) jointly shape representations and cluster assignments. The results demonstrate substantial improvements over RNN- and Transformer-based methods, with strong scalability and practical impact for real-world LSTC tasks, while also providing insights into Transformer limitations for univariate long-horizon clustering.

Abstract

Time series clustering is fundamental in data analysis for discovering temporal patterns. Despite recent advancements, learning cluster-friendly representations is still challenging, particularly with long and complex time series. Deep temporal clustering methods have been trying to integrate the canonical k-means into end-to-end training of neural networks but fall back on surrogate losses due to the non-differentiability of the hard cluster assignment, yielding sub-optimal solutions. In addition, the autoregressive strategy used in the state-of-the-art RNNs is subject to error accumulation and slow training, while recent research findings have revealed that Transformers are less effective due to time points lacking semantic meaning, to the permutation invariance of attention that discards the chronological order and high computation cost. In light of these observations, we present LoSTer which is a novel dense autoencoder architecture for the long-sequence time series clustering problem (LSTC) capable of optimizing the k-means objective via the Gumbel-softmax reparameterization trick and designed specifically for accurate and fast clustering of long time series. Extensive experiments on numerous benchmark datasets and two real-world applications prove the effectiveness of LoSTer over state-of-the-art RNNs and Transformer-based deep clustering methods.

Figures (2)

• Figure 1: The overall architecture of LoSTer for long-sequence time series clustering (LSTC). It employs a two-view architecture with identical autoencoders for reconstructing original and augmented time series. Both the encoder and decoder are instantiated as a stack of dense residual blocks. The latent representations of data and centroids and the cluster assignments are learned end-to-end through k-means and dual contrastive loss.
• Figure 2: The bar chart reports the actual training times [s] by methods for one epoch referred to the Store Sales dataset. The y-axis is plotted in log-scale. The fastest IDEC and slowest DTCR are distanced by 3 orders of magnitude.