Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TS2Vec: Towards Universal Representation of Time Series

Zhihan Yue, Yujing Wang, Juanyong Duan, Tianmeng Yang, Congrui Huang, Yunhai Tong, Bixiong Xu

TL;DR

The paper tackles the challenge of learning universal time-series representations at arbitrary semantic levels. It introduces TS2Vec, a universal framework that performs hierarchical contrastive learning over augmented context views to produce per-timestamp embeddings and flexible sub-series representations via pooling. Key contributions include contextual consistency for positive pair selection, timestamp masking, random cropping, and dual temporal/instance-wise losses across multiple scales, leading to state-of-the-art results in classification, forecasting, and anomaly detection. Empirical results demonstrate strong performance, efficiency, and robustness to missing data, highlighting TS2Vec's practical impact for diverse time-series tasks.

Abstract

This paper presents TS2Vec, a universal framework for learning representations of time series in an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature. The source code is publicly available at https://github.com/yuezhihan/ts2vec.

TS2Vec: Towards Universal Representation of Time Series

TL;DR

The paper tackles the challenge of learning universal time-series representations at arbitrary semantic levels. It introduces TS2Vec, a universal framework that performs hierarchical contrastive learning over augmented context views to produce per-timestamp embeddings and flexible sub-series representations via pooling. Key contributions include contextual consistency for positive pair selection, timestamp masking, random cropping, and dual temporal/instance-wise losses across multiple scales, leading to state-of-the-art results in classification, forecasting, and anomaly detection. Empirical results demonstrate strong performance, efficiency, and robustness to missing data, highlighting TS2Vec's practical impact for diverse time-series tasks.

Abstract

This paper presents TS2Vec, a universal framework for learning representations of time series in an arbitrary semantic level. Unlike existing methods, TS2Vec performs contrastive learning in a hierarchical way over augmented context views, which enables a robust contextual representation for each timestamp. Furthermore, to obtain the representation of an arbitrary sub-sequence in the time series, we can apply a simple aggregation over the representations of corresponding timestamps. We conduct extensive experiments on time series classification tasks to evaluate the quality of time series representations. As a result, TS2Vec achieves significant improvement over existing SOTAs of unsupervised time series representation on 125 UCR datasets and 29 UEA datasets. The learned timestamp-level representations also achieve superior results in time series forecasting and anomaly detection tasks. A linear regression trained on top of the learned representations outperforms previous SOTAs of time series forecasting. Furthermore, we present a simple way to apply the learned representations for unsupervised anomaly detection, which establishes SOTA results in the literature. The source code is publicly available at https://github.com/yuezhihan/ts2vec.

Paper Structure

This paper contains 44 sections, 1 theorem, 7 equations, 9 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Method
  3. Problem Definition
  4. Model Architecture
  5. Contextual Consistency
  6. Timestamp Masking
  7. Random Cropping
  8. Hierarchical Contrasting
  9. Temporal Contrastive Loss
  10. Instance-wise Contrastive Loss
  11. Experiments
  12. Time Series Classification
  13. Time Series Forecasting
  14. Time Series Anomaly Detection
  15. Analysis
  16. ...and 29 more sections

Key Result

Lemma B.1

Given $W\in \mathbbm{R}^{F'\times F}$ and $F'>F$, there exists $b\in \mathbbm{R}^{F'}$ such that $Wx+b\neq 0$ for all $x\in \mathbbm{R}^{F}$.

Figures (9)

  • Figure 1: The proposed architecture of TS2Vec. Although this figure shows a univariate time series as the input example, the framework supports multivariate input. Each parallelogram denotes the representation vector on a timestamp of an instance.
  • Figure 2: Positive pair selection strategies.
  • Figure 3: Two typical cases of the distribution change of time series, with the heatmap visualization of the learned representations over time using subseries consistency and temporal consistency respectively.
  • Figure 4: Critical Difference (CD) diagram of representation learning methods on time series classification tasks with a confidence level of 95%.
  • Figure 5: A prediction slice (H=336) of TS2Vec, Informer and TCN on the test set of ETTh$_2$.
  • ...and 4 more figures

Theorems & Definitions (2)

  • Lemma B.1
  • proof