Parametric Augmentation for Time Series Contrastive Learning

TL;DR

The paper addresses the challenge of designing effective augmentations for time series contrastive learning by introducing AutoTCL, a parametric augmentation framework that factorizes each instance into an informative component $x^*$ and a task-irrelevant part $\Delta x$, then learns a lossless, adaptive view via an invertible transform $g$ and a learnable mask $h$. A principled objective based on the Principle of Relevant Information (PRI) guides the augmentation network, balancing information preservation with view diversity, while a time-series encoder trains with both global and local contrastive losses. Empirical results on forecasting and classification demonstrate consistent improvements over strong baselines, with univariate forecasting improved by around 6.5% in MSE and 4.8% in MAE, and classification gains of about 1.2% in average accuracy. The approach is encoder-agnostic and shows robust benefits across multiple backbones and datasets, highlighting the practical impact of adaptive, factorization-based augmentations for time series representations.

Abstract

Modern techniques like contrastive learning have been effectively used in many areas, including computer vision, natural language processing, and graph-structured data. Creating positive examples that assist the model in learning robust and discriminative representations is a crucial stage in contrastive learning approaches. Usually, preset human intuition directs the selection of relevant data augmentations. Due to patterns that are easily recognized by humans, this rule of thumb works well in the vision and language domains. However, it is impractical to visually inspect the temporal structures in time series. The diversity of time series augmentations at both the dataset and instance levels makes it difficult to choose meaningful augmentations on the fly. In this study, we address this gap by analyzing time series data augmentation using information theory and summarizing the most commonly adopted augmentations in a unified format. We then propose a contrastive learning framework with parametric augmentation, AutoTCL, which can be adaptively employed to support time series representation learning. The proposed approach is encoder-agnostic, allowing it to be seamlessly integrated with different backbone encoders. Experiments on univariate forecasting tasks demonstrate the highly competitive results of our method, with an average 6.5\% reduction in MSE and 4.7\% in MAE over the leading baselines. In classification tasks, AutoTCL achieves a $1.2\%$ increase in average accuracy.

Figures (4)

• Figure 1: The framework of our AutoTCL. The augmentation network extracts the informative part from the original instance and losslessly transforms it to $v^*$. The encoder network is optimized with the contrastive objective.
• Figure 2: T-SNE visualization of different augmentation instances. In samples $a$ and $b$, AutoTCL-generated samples are closer to the original instance $x$ than other instances $x'$ with large variety
• Figure 3: The augmentation loss, Eq. (\ref{['eq:augobj']}) and contrastive loss, Eq. (\ref{['eq:contrasive']}), in the training process
• Figure 4: Parameter sensitivity studies on ETTh$_1$.

Theorems & Definitions (3)

• proof
• proof
• proof