Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Simple Contrastive Representation Learning for Time Series Forecasting

Xiaochen Zheng, Xingyu Chen, Manuel Schürch, Amina Mollaysa, Ahmed Allam, Michael Krauthammer

TL;DR

SimTS, a simple representation learning approach for improving time series forecasting by learning to predict the future from the past in the latent space, exclusively uses positive pairs and does not depend on negative pairs or specific characteristics of a given time series.

Abstract

Contrastive learning methods have shown an impressive ability to learn meaningful representations for image or time series classification. However, these methods are less effective for time series forecasting, as optimization of instance discrimination is not directly applicable to predicting the future state from the historical context. To address these limitations, we propose SimTS, a simple representation learning approach for improving time series forecasting by learning to predict the future from the past in the latent space. SimTS exclusively uses positive pairs and does not depend on negative pairs or specific characteristics of a given time series. In addition, we show the shortcomings of the current contrastive learning framework used for time series forecasting through a detailed ablation study. Overall, our work suggests that SimTS is a promising alternative to other contrastive learning approaches for time series forecasting.

Simple Contrastive Representation Learning for Time Series Forecasting

TL;DR

SimTS, a simple representation learning approach for improving time series forecasting by learning to predict the future from the past in the latent space, exclusively uses positive pairs and does not depend on negative pairs or specific characteristics of a given time series.

Abstract

Contrastive learning methods have shown an impressive ability to learn meaningful representations for image or time series classification. However, these methods are less effective for time series forecasting, as optimization of instance discrimination is not directly applicable to predicting the future state from the historical context. To address these limitations, we propose SimTS, a simple representation learning approach for improving time series forecasting by learning to predict the future from the past in the latent space. SimTS exclusively uses positive pairs and does not depend on negative pairs or specific characteristics of a given time series. In addition, we show the shortcomings of the current contrastive learning framework used for time series forecasting through a detailed ablation study. Overall, our work suggests that SimTS is a promising alternative to other contrastive learning approaches for time series forecasting.
Paper Structure (22 sections, 4 equations, 4 figures, 7 tables, 1 algorithm)

This paper contains 22 sections, 4 equations, 4 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methods
  4. Motivation
  5. SimTS: Simple Representation Learning for Time Series Forecasting
  6. Multi-Scale Encoder
  7. Stop-gradient Operation
  8. Experiments
  9. Datasets and Baselines
  10. Experimental setup
  11. Results
  12. Ablation Study
  13. Backbones
  14. Negative Samples
  15. Stop-Gradient Operation
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Problems with selecting negative pairs based on methods proposed in nclyue2022ts2vecwoo2022cost when cross-instance and cross-time repeated patterns exist.
  • Figure 2: Illustration of our proposed SimTS.
  • Figure 3: Multi-scale encoder. Composed of a projection layer and a set of parallel 1d convolutions with kernel size $2^i$, for $i \in \{0,1,...,m\}$. An averaged pooling layer is added on the top of convolutions.
  • Figure 4: Ablation study of stop-gradient operation. (a) SimTS architecture. (b) SimTS without stop-gradient operation. (c) RevSimTS with stop-gradient on the history encoding path.