Efficient Hierarchical Contrastive Self-supervising Learning for Time Series Classification via Importance-aware Resolution Selection

TL;DR

This work tackles the high computational cost of hierarchical contrastive self-supervised learning for long time series. It introduces an importance-aware resolution selection mechanism that trains the model on a single, most informative resolution per epoch, reducing computation while preserving representation quality. The method combines crops and time-step masking, a 1-D ResNet18 encoder, and a dual loss (temporal and instance) at each resolution, with a principled softmax-based scoring and sampling strategy to select the training target. Across ten UEA/UCR datasets, the approach achieves training-time savings with accuracy on par with or better than the original hierarchical framework and strong performance relative to non-SSL baselines, highlighting its practicality for large-scale time-series tasks.

Abstract

Recently, there has been a significant advancement in designing Self-Supervised Learning (SSL) frameworks for time series data to reduce the dependency on data labels. Among these works, hierarchical contrastive learning-based SSL frameworks, which learn representations by contrasting data embeddings at multiple resolutions, have gained considerable attention. Due to their ability to gather more information, they exhibit better generalization in various downstream tasks. However, when the time series data length is significant long, the computational cost is often significantly higher than that of other SSL frameworks. In this paper, to address this challenge, we propose an efficient way to train hierarchical contrastive learning models. Inspired by the fact that each resolution's data embedding is highly dependent, we introduce importance-aware resolution selection based training framework to reduce the computational cost. In the experiment, we demonstrate that the proposed method significantly improves training time while preserving the original model's integrity in extensive time series classification performance evaluations. Our code could be found here, https://github.com/KEEBVIN/IARS

Figures (7)

• Figure 1: Proposed Framework: Starting with original and augmented time series $T$ and $T'$, the data is encoded into representations $z$ and $z'$. The framework computes loss across multiple resolutions using Temporal Contrastive methods combined with 1-Dimensional MaxPooling. The framework calculates losses $\mathcal{L}_1$ to $\mathcal{L}_N$ across multiple resolutions. Each loss is associated with a score that evaluates its significance. Based on these scores, the importance-aware sampling decides which resolution's loss to focus on for a specific training epoch.
• Figure 2: Temporal Wise Loss
• Figure 3: Instance Wise Loss
• Figure 4: Importance-aware Resolution Selection Overview: (a): aligned resolution loss $\mathcal{L}$. (b): Every absented loss value is replaced by its prior recorded loss $\mathcal{L}'$. (c): Each loss is passed through Equation 10 and generates a probability. (d):Sampling from the generated distribution.
• Figure 5: Execution time and Accuracy comparison against Original Hierarchical Contrastive Learning Framework with Differing Embedding Size