Learning Time-Series Representations by Hierarchical Uniformity-Tolerance Latent Balancing

TL;DR

This work tackles unsupervised time-series representation learning by balancing uniformity and tolerance in a hierarchical contrastive framework. TimeHUT introduces a periodic temperature scheduler $\tau(\sigma)$ and a hierarchical angular margin loss to jointly optimize temporal and instance-wise relations, enabling robust representations across time. Built on TS2Vec, it combines hierarchical temporal/institution-wise contrasts with the total loss $L_{Total}=L_{HierSch}+L_{HierAng}$ and demonstrates improvements on large-scale benchmarks. Experiments on 128 UCR and 30 UEA datasets plus Yahoo/KPI show strong classification performance and competitive anomaly detection, with released code to support reproducibility.

Abstract

We propose TimeHUT, a novel method for learning time-series representations by hierarchical uniformity-tolerance balancing of contrastive representations. Our method uses two distinct losses to learn strong representations with the aim of striking an effective balance between uniformity and tolerance in the embedding space. First, TimeHUT uses a hierarchical setup to learn both instance-wise and temporal information from input time-series. Next, we integrate a temperature scheduler within the vanilla contrastive loss to balance the uniformity and tolerance characteristics of the embeddings. Additionally, a hierarchical angular margin loss enforces instance-wise and temporal contrast losses, creating geometric margins between positive and negative pairs of temporal sequences. This approach improves the coherence of positive pairs and their separation from the negatives, enhancing the capture of temporal dependencies within a time-series sample. We evaluate our approach on a wide range of tasks, namely 128 UCR and 30 UAE datasets for univariate and multivariate classification, as well as Yahoo and KPI datasets for anomaly detection. The results demonstrate that TimeHUT outperforms prior methods by considerable margins on classification, while obtaining competitive results for anomaly detection. Finally, detailed sensitivity and ablation studies are performed to evaluate different components and hyperparameters of our method.

Figures (4)

• Figure 1: Overview of TimeHUT architecture. (a) Input time-series $X$ (with samples $i$, $j$, and $k$ shown in different colors) are randomly cropped into two overlapping subseries. (b) The encoder processes these subseries, followed by hierarchical contrasting at both temporal and instance-wise levels through max pooling, computing both temporal contrastive (within-sample across time) and instance-wise contrastive losses (across samples). (c) The temperature scheduler $\tau(\sigma)$ dynamically balances uniformity-tolerance trade-offs ($L_{HierSch}$), while angular margin loss enforces geometric separation between positive and negative pairs ($L_{HierAng}$). The final loss combines both components.
• Figure 2: The accuracy of TimeHUT versus different values of $c_i$ and $c_t$, on the "Star Light Curves" dataset (top left), "China Town" dataset (top right), "Ford-A" dataset (bottom left), and "None-Invasive Fetal ECG Thorax 1" dataset (bottom right).
• Figure 3: Uniformity and tolerance values for different datasets.
• Figure A1: UMAP visualization of the learned representations by TimeHUT (right) vs. TS2Vec (left) on "Basic Motion" (top row) and "Dodger Loop Weekend" (bottom row) datasets.