Semi-Supervised End-To-End Contrastive Learning For Time Series Classification

TL;DR

The results demonstrate that SLOTS is a simple yet effective framework, and when compared to the two-stage framework, the end-to-end SLOTS utilizes the same input data, consumes a similar computational cost, but delivers significantly improved performance.

Abstract

Time series classification is a critical task in various domains, such as finance, healthcare, and sensor data analysis. Unsupervised contrastive learning has garnered significant interest in learning effective representations from time series data with limited labels. The prevalent approach in existing contrastive learning methods consists of two separate stages: pre-training the encoder on unlabeled datasets and fine-tuning the well-trained model on a small-scale labeled dataset. However, such two-stage approaches suffer from several shortcomings, such as the inability of unsupervised pre-training contrastive loss to directly affect downstream fine-tuning classifiers, and the lack of exploiting the classification loss which is guided by valuable ground truth. In this paper, we propose an end-to-end model called SLOTS (Semi-supervised Learning fOr Time clasSification). SLOTS receives semi-labeled datasets, comprising a large number of unlabeled samples and a small proportion of labeled samples, and maps them to an embedding space through an encoder. We calculate not only the unsupervised contrastive loss but also measure the supervised contrastive loss on the samples with ground truth. The learned embeddings are fed into a classifier, and the classification loss is calculated using the available true labels. The unsupervised, supervised contrastive losses and classification loss are jointly used to optimize the encoder and classifier. We evaluate SLOTS by comparing it with ten state-of-the-art methods across five datasets. The results demonstrate that SLOTS is a simple yet effective framework. When compared to the two-stage framework, our end-to-end SLOTS utilizes the same input data, consumes a similar computational cost, but delivers significantly improved performance. We release code and datasets at https://anonymous.4open.science/r/SLOTS-242E.

Figures (2)

• Figure 1: (a) Illustration of a standard self-supervised contrastive model for classification tasks. The whole framework includes two stages: pre-training and fine-tuning. (b) Proposed end-to-end framework. Our model exhibits two properties that are unattainable in conventional two-stage frameworks: 1) the unsupervised contrastive loss, calculated on unlabeled data, contributes to the optimization of the classifier, enabling more effective utilization of the unlabeled data; 2) the incorporation of a newly added supervised contrastive loss further enhances performance and facilitates more efficient learning, resulting in a more powerful and robust model.
• Figure 2: Overview of the proposed SLOTS. Our end-to-end model accepts a semi-labeled dataset consisting of unlabeled time series and labeled time series samples. We perform augmentations on the unlabeled sample $\bm{x}^{\textsc{u}}$, generating two views $\bm{x}^{\textsc{u}}_{i}$ and $\bm{x}^{\textsc{u}}_{j}$. These two views are fed into an encoder $f$ that maps samples to a latent embedding space, where we compute the unsupervised contrastive loss $\mathcal{L}_u$ through instance discrimination. Concurrently, we evaluate the supervised loss $\mathcal{L}s$ on the representations learned from the labeled sample $\bm{x}^{\textsc{l}}$. The $\bm{z}^{\textsc{l}}_+$ associates with the same label as $\bm{z}^{\textsc{l}}$ while $\bm{z}^{\textsc{l}}_-$ belongs to a different label. Additionally, we calculate the classification loss $\mathcal{L}_c$ on the labeled samples. The model is optimized using a hybrid loss $\mathcal{L}$.