Time-Series Representation Learning via Temporal and Contextual Contrasting

TL;DR

This work tackles the challenge of learning meaningful representations from unlabeled time-series data by proposing TS-TCC, a self-supervised framework that uses two correlated views generated by weak and strong augmentations. It introduces a temporal contrasting module with cross-view future prediction and a contextual contrasting module to sharpen discriminability, achieving strong linear, semi-supervised, and transfer performance across HAR, Sleep-EDF, Epilepsy, and fault-diagnosis datasets. The results demonstrate that representations learned by TS-TCC can rival supervised training with far fewer labels and exhibit good cross-domain transfer, highlighting its practical impact for real-world time-series analysis.

Abstract

Learning decent representations from unlabeled time-series data with temporal dynamics is a very challenging task. In this paper, we propose an unsupervised Time-Series representation learning framework via Temporal and Contextual Contrasting (TS-TCC), to learn time-series representation from unlabeled data. First, the raw time-series data are transformed into two different yet correlated views by using weak and strong augmentations. Second, we propose a novel temporal contrasting module to learn robust temporal representations by designing a tough cross-view prediction task. Last, to further learn discriminative representations, we propose a contextual contrasting module built upon the contexts from the temporal contrasting module. It attempts to maximize the similarity among different contexts of the same sample while minimizing similarity among contexts of different samples. Experiments have been carried out on three real-world time-series datasets. The results manifest that training a linear classifier on top of the features learned by our proposed TS-TCC performs comparably with the supervised training. Additionally, our proposed TS-TCC shows high efficiency in few-labeled data and transfer learning scenarios. The code is publicly available at https://github.com/emadeldeen24/TS-TCC.

Figures (4)

• Figure 1: Overall architecture of proposed TS-TCC model.
• Figure 2: Architecture of Transformer model used in Temporal Contrasting module. The token $c$ in the output is sent next to the Contextual Contrasting module.
• Figure 3: Comparison between supervised training vs. TS-TCC fine-tuning for different few-labeled data scenarios in terms of MF1.
• Figure 4: Three sensitivity analysis experiments on HAR dataset.