Fuzzy Cluster-Aware Contrastive Clustering for Time Series

TL;DR

FCACC addresses unsupervised time-series clustering by jointly optimizing representation learning and clustering through a fuzzy clustering framework. It introduces a three-view data augmentation strategy, a cluster-aware hard negative sampling mechanism, and a cluster-awareness generation module to guide the joint optimization, with a pretraining stage followed by two-phase training. Evaluated on 40 UCR datasets, FCACC outperforms eight baselines in both NMI and RI, demonstrating robust clustering of complex time-series patterns. The method advances self-supervised time-series analysis by integrating cluster structure into representation learning and sampling, with potential extensions to multi-modal or streaming data.

Abstract

The rapid growth of unlabeled time series data, driven by the Internet of Things (IoT), poses significant challenges in uncovering underlying patterns. Traditional unsupervised clustering methods often fail to capture the complex nature of time series data. Recent deep learning-based clustering approaches, while effective, struggle with insufficient representation learning and the integration of clustering objectives. To address these issues, we propose a fuzzy cluster-aware contrastive clustering framework (FCACC) that jointly optimizes representation learning and clustering. Our approach introduces a novel three-view data augmentation strategy to enhance feature extraction by leveraging various characteristics of time series data. Additionally, we propose a cluster-aware hard negative sample generation mechanism that dynamically constructs high-quality negative samples using clustering structure information, thereby improving the model's discriminative ability. By leveraging fuzzy clustering, FCACC dynamically generates cluster structures to guide the contrastive learning process, resulting in more accurate clustering. Extensive experiments on 40 benchmark datasets show that FCACC outperforms the selected baseline methods (eight in total), providing an effective solution for unsupervised time series learning.

Figures (8)

• Figure 1: Overall Structure of FCACC Framework. This diagram illustrates the complete FCACC framework, which is organized into a pre-training stage and a joint optimization stage. The system comprises three primary modules: (1) the contrastive learning module, which enhances representation quality via multi-view augmentation and hard negative sample generation; (2) the fuzzy clustering module, which effectively models complex membership relationships in the data; and (3) the cluster-awareness generation module, which dynamically constructs cluster structures that steer the joint optimization process. The directional arrows denote the data flow among the modules, emphasizing their collaborative interaction through the cluster-awareness mechanism.
• Figure 2: Data Augmentation Process. The input raw time series is processed through multiple random cropping operations to generate three subsequences as shown in the figure. These subsequences share the same overlapping region $[n_1, m_2]$. Among them, the first time segment $[n_1, m_2]$ undergoes additional perturbations to form $\boldsymbol{X}^{(a)}$, while the latter two segments remain unchanged, forming $\boldsymbol{X}^{(b)}$ and $\boldsymbol{X}^{(c)}$, respectively.
• Figure 3: Cluster-aware Hard Negative Sample Generation and its Effect Comparison. The \ref{['sub@subfig:hard_a']} illustrates the process of generating cluster-aware hard negative samples. We generate these samples by mixing time series from different clusters at a certain ratio. These samples do not belong to any original cluster but are close to the boundary of the positive sample region, thus providing more challenging learning signals. The \ref{['sub@subfig:hard_b']} shows an example of the distribution of samples in the embedding space from a portion of the GesturePebbleZ1 training set in the UCR archive. For each positive anchor (red square), the original negative samples (gray squares) include many easy negative samples (gray squares far from the positive samples) and a few same-cluster samples (gray squares close to the positive sample). Mixing positive samples from the same cluster results in hard negative samples (green triangles) that are very similar to the positive samples. In contrast, cluster-aware hard negative samples (blue triangles) avoid misclassifying same-cluster positive samples as negative samples.
• Figure 4: Variation in Cluster-Aware Samples During Training. The model's progressively enhanced understanding of the cluster structure is reflected in this change.
• Figure 5: Effect of Parameters.