AimTS: Augmented Series and Image Contrastive Learning for Time Series Classification

TL;DR

AimTS tackles data scarcity and cross-domain generalization in time series classification by pre-training on multi-source data with a two-level prototype-based contrastive loss and by incorporating an image modality through series-image contrastive learning. The method combines intra- and inter-prototype losses to robustly utilize diverse augmentations, while a geodesic mixup strategy enriches negative samples by jointly encoding numerical and structural information. Empirical results on UCR and UEA benchmarks show strong generalization across datasets, with significant gains in few-shot settings and substantial efficiency in memory and computation. Overall, AimTS advances cross-domain TSC by leveraging multi-source pre-training and cross-modal cues to achieve robust, scalable representations with practical impact for diverse time series tasks.

Abstract

Time series classification (TSC) is an important task in time series analysis. Existing TSC methods mainly train on each single domain separately, suffering from a degradation in accuracy when the samples for training are insufficient in certain domains. The pre-training and fine-tuning paradigm provides a promising direction for solving this problem. However, time series from different domains are substantially divergent, which challenges the effective pre-training on multi-source data and the generalization ability of pre-trained models. To handle this issue, we introduce Augmented Series and Image Contrastive Learning for Time Series Classification (AimTS), a pre-training framework that learns generalizable representations from multi-source time series data. We propose a two-level prototype-based contrastive learning method to effectively utilize various augmentations in multi-source pre-training, which learns representations for TSC that can be generalized to different domains. In addition, considering augmentations within the single time series modality are insufficient to fully address classification problems with distribution shift, we introduce the image modality to supplement structural information and establish a series-image contrastive learning to improve the generalization of the learned representations for TSC tasks. Extensive experiments show that after multi-source pre-training, AimTS achieves good generalization performance, enabling efficient learning and even few-shot learning on various downstream TSC datasets.

Figures (15)

• Figure 1: Illustration of existing deep learning methods for TSC.
• Figure 2: The example of augmentation causing semantic changes. The ECG200 dataset records electrocardiograms for healthy and myocardial infarction (MI) patients. (a) Pattern illustrations of two labels. T wave inversion is a sign of MI. (b) The black line shows a normal ECG and the green line shows an MI ECG. (c) The blue dashed line shows jitter augmentation on the normal ECG and the T wave of this augmented sample has been inverted. (d) After jittering, the normal sample becomes more similar to the MI sample, leading to a change in its semantics.
• Figure 3: Overview of AimTS.
• Figure 4: Overview of prototype-based contrastive learning. Solid and dashed shapes of the same color and the same shape represent the representations of the two augmented views from the same augmentation on the same sample. Different colors indicate different augmentation methods. Different shapes represent distinct samples. Gray shapes denote prototypes for each sample, with two prototypes generated per sample. (a) Perform G types of augmentations on time series data by applying each twice to extract representations. (b) Illustration of prototype-based contrastive learning. The intra-prototype contrastive learning is conducted within the same sample and its augmentations. The color band indicates the distance between representations of different augmentations, where transitioning from white to red indicates the increase of their distance, and the decrease of the temperature parameter $\tau$ accordingly. The inter-prototype contrastive learning is conducted across different samples.
• Figure 5: Geodesic mixup strategy and mixup contrastive learning. (a) Illustration of mixup contrastive learning. Various green shapes, such as stars, triangles, and others, represent the time series representations, while purple shapes indicate their corresponding image representations. (b) Illustration of geodesic mixup. The green and purple squares represent the time series and image representations on the hyperspherical surface, respectively. The green-purple square represents the mixed representation by geodesic mixup, which can be seen to remain on the hypersphere.