StatioCL: Contrastive Learning for Time Series via Non-Stationary and Temporal Contrast

TL;DR

Evaluated on real-world benchmark time series classification datasets, StatioCL demonstrates a substantial improvement over state-of-the-art CL methods, achieving a 2.9% increase in Recall and a 19.2% reduction in FNPs.

Abstract

Contrastive learning (CL) has emerged as a promising approach for representation learning in time series data by embedding similar pairs closely while distancing dissimilar ones. However, existing CL methods often introduce false negative pairs (FNPs) by neglecting inherent characteristics and then randomly selecting distinct segments as dissimilar pairs, leading to erroneous representation learning, reduced model performance, and overall inefficiency. To address these issues, we systematically define and categorize FNPs in time series into semantic false negative pairs and temporal false negative pairs for the first time: the former arising from overlooking similarities in label categories, which correlates with similarities in non-stationarity and the latter from neglecting temporal proximity. Moreover, we introduce StatioCL, a novel CL framework that captures non-stationarity and temporal dependency to mitigate both FNPs and rectify the inaccuracies in learned representations. By interpreting and differentiating non-stationary states, which reflect the correlation between trends or temporal dynamics with underlying data patterns, StatioCL effectively captures the semantic characteristics and eliminates semantic FNPs. Simultaneously, StatioCL establishes fine-grained similarity levels based on temporal dependencies to capture varying temporal proximity between segments and to mitigate temporal FNPs. Evaluated on real-world benchmark time series classification datasets, StatioCL demonstrates a substantial improvement over state-of-the-art CL methods, achieving a 2.9% increase in Recall and a 19.2% reduction in FNPs. Most importantly, StatioCL also shows enhanced data efficiency and robustness against label scarcity.

Figures (7)

• Figure 1: Identification of False Negative Pairs in Time Series Contrastive Learning Due to Random Selection: (a) Semantic FNPs, occurring when the selection process ignores the similarity between labels and (b) Temporal FNPs, resulting from neglecting the similarity in terms of temporal proximity.
• Figure 2: StatioCL Framework Overview: Input sequence $X$ is first passed through (1) Non-stationarity Assessment module to get the non-stationary state for each segment. (2) Augmentation module will then generate two augmentation views and encode augmented views into feature space. After that, each view will be passed to (3) Non-stationary Contrast module to construct negative pairs based on non-stationary states and reduce semantic FNPs. (4) Temporal Contrast module to create weighted negative pairs based on time differences and alleviate temporal FNPs. Finally, the overall loss $L$ is calculated by combining $L_{NC}$ and $L_{TC}$.
• Figure 3: Example of Stationary and Non-Stationary ECG Signal. If p-value $<$ 0.01, the stationary state $l_i$ is set to 0; otherwise, $l_i$ is assigned a value of 1.
• Figure 4: Weight Distribution for Soft-Negative Pairs: The histogram depicts the assigned weights to negative pairs based on their time difference, following the beta distribution.
• Figure 5: Learned Representation Embedding Spaces on the Epilepsy Training Set.