Soft Contrastive Learning for Time Series

TL;DR

SoftCLT addresses the misalignment in traditional contrastive learning for time series by introducing soft, data-space–driven assignments for both instance-wise and temporal relationships. The method defines soft weights based on a distance $D(\cdot,\cdot)$ (favoring DTW) and timestamp differences, and optimizes a joint loss that generalizes InfoNCE while preserving intra-series correlations. It combines soft instance-wise contrastive loss and soft temporal contrastive loss, with hierarchical temporal representations and a tunable balance parameter $\lambda$, yielding substantial gains across classification, semi-supervised learning, transfer learning, and anomaly detection. The approach is plug-and-play, empirically robust to seasonality and distribution shifts, and can be integrated with existing TS-CL frameworks, achieving state-of-the-art performance on numerous benchmarks.

Abstract

Contrastive learning has shown to be effective to learn representations from time series in a self-supervised way. However, contrasting similar time series instances or values from adjacent timestamps within a time series leads to ignore their inherent correlations, which results in deteriorating the quality of learned representations. To address this issue, we propose SoftCLT, a simple yet effective soft contrastive learning strategy for time series. This is achieved by introducing instance-wise and temporal contrastive loss with soft assignments ranging from zero to one. Specifically, we define soft assignments for 1) instance-wise contrastive loss by the distance between time series on the data space, and 2) temporal contrastive loss by the difference of timestamps. SoftCLT is a plug-and-play method for time series contrastive learning that improves the quality of learned representations without bells and whistles. In experiments, we demonstrate that SoftCLT consistently improves the performance in various downstream tasks including classification, semi-supervised learning, transfer learning, and anomaly detection, showing state-of-the-art performance. Code is available at this repository: https://github.com/seunghan96/softclt.

Figures (16)

• Figure 1: Overall framework of SoftCLT. Unlike the conventional hard CL that gives either positive or negative assignments to sample pairs, SoftCLT gives soft assignments to both instance-wise and temporal relationships. Two views of the same sample are denoted as $r$ and $\tilde{r}$, respectively.
• Figure 2: (a) shows examples of soft assignments for soft temporal CL, where a smaller $\tau_T$ results in smoother assignments. (b) is an example of hierarchical representations, demonstrating that increasing layer depth results in a larger semantic difference between adjacent time steps, so $\tau_{T}$ should be increased to compensate for it.
• Figure 3: Accuracy and rank on UCR/UEA.
• Figure 4: CD diagram on UCR/UEA.
• Figure 5: Comparison of soft CL methods.