Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rank Supervised Contrastive Learning for Time Series Classification

Qianying Ren, Dongsheng Luo, Dongjin Song

TL;DR

Thoroughly empirical studies on 128 UCR and 30 UEA datasets demonstrate that the proposed RankSCL can achieve state-of-the-art performance compared to existing baseline methods.

Abstract

Recently, various contrastive learning techniques have been developed to categorize time series data and exhibit promising performance. A general paradigm is to utilize appropriate augmentations and construct feasible positive samples such that the encoder can yield robust and discriminative representations by mapping similar data points closer together in the feature space while pushing dissimilar data points farther apart. Despite its efficacy, the fine-grained relative similarity (e.g., rank) information of positive samples is largely ignored, especially when labeled samples are limited. To this end, we present Rank Supervised Contrastive Learning (RankSCL) to perform time series classification. Different from conventional contrastive learning frameworks, RankSCL augments raw data in a targeted way in the embedding space and adopts certain filtering rules to select more informative positive and negative pairs of samples. Moreover, a novel rank loss is developed to assign different weights for different levels of positive samples, enable the encoder to extract the fine-grained information of the same class, and produce a clear boundary among different classes. Thoroughly empirical studies on 128 UCR datasets and 30 UEA datasets demonstrate that the proposed RankSCL can achieve state-of-the-art performance compared to existing baseline methods.

Rank Supervised Contrastive Learning for Time Series Classification

TL;DR

Thoroughly empirical studies on 128 UCR and 30 UEA datasets demonstrate that the proposed RankSCL can achieve state-of-the-art performance compared to existing baseline methods.

Abstract

Recently, various contrastive learning techniques have been developed to categorize time series data and exhibit promising performance. A general paradigm is to utilize appropriate augmentations and construct feasible positive samples such that the encoder can yield robust and discriminative representations by mapping similar data points closer together in the feature space while pushing dissimilar data points farther apart. Despite its efficacy, the fine-grained relative similarity (e.g., rank) information of positive samples is largely ignored, especially when labeled samples are limited. To this end, we present Rank Supervised Contrastive Learning (RankSCL) to perform time series classification. Different from conventional contrastive learning frameworks, RankSCL augments raw data in a targeted way in the embedding space and adopts certain filtering rules to select more informative positive and negative pairs of samples. Moreover, a novel rank loss is developed to assign different weights for different levels of positive samples, enable the encoder to extract the fine-grained information of the same class, and produce a clear boundary among different classes. Thoroughly empirical studies on 128 UCR datasets and 30 UEA datasets demonstrate that the proposed RankSCL can achieve state-of-the-art performance compared to existing baseline methods.
Paper Structure (21 sections, 4 equations, 6 figures, 4 tables, 1 algorithm)

This paper contains 21 sections, 4 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contrastive Time Series Representation Learning
  4. Time Series Classification
  5. Time Series Data Augmentation
  6. Method
  7. Notations and Problem Definition
  8. Framework
  9. Model Architecture.
  10. Data Augmentation on the Embedding Space
  11. Selection of Valid Triplet Pairs
  12. Rank Supervised Contrastive Learning Loss
  13. Algorithms
  14. Experiments
  15. Datasets and Baselines
  16. ...and 6 more sections

Figures (6)

  • Figure 1: Overview of RankSCL framework, consisting of three components: (1) a Fully Convolutional Network that captures the embeddings of raw time series instances, (2) targeted data augmentation that generates more samples in the embedding space, (3) selection of valid triplets and calculation of rank loss to train the encoder network. Even though this figure shows a univariate time series instance as an example, the architecture supports multivariate instances.
  • Figure 2: Valid Triplet Selection
  • Figure 3: Properties of rank function $R(\cdot)$
  • Figure 4: Critical Difference (CD) diagram of Univariate Time series classification task
  • Figure 5: Critical Difference (CD) diagram of Multivariate Time series classification task
  • ...and 1 more figures