Higher-order Cross-structural Embedding Model for Time Series Analysis

TL;DR

High-TS is a novel framework that jointly models both temporal and spatial perspectives by combining multiscale Transformer with Topological Deep Learning (TDL), and utilizes contrastive learning to integrate these two structures for generating robust and discriminative representations.

Abstract

Time series analysis has gained significant attention due to its critical applications in diverse fields such as healthcare, finance, and sensor networks. The complexity and non-stationarity of time series make it challenging to capture the interaction patterns across different timestamps. Current approaches struggle to model higher-order interactions within time series, and focus on learning temporal or spatial dependencies separately, which limits performance in downstream tasks. To address these gaps, we propose Higher-order Cross-structural Embedding Model for Time Series (High-TS), a novel framework that jointly models both temporal and spatial perspectives by combining multiscale Transformer with Topological Deep Learning (TDL). Meanwhile, High-TS utilizes contrastive learning to integrate these two structures for generating robust and discriminative representations. Extensive experiments show that High-TS outperforms state-of-the-art methods in various time series tasks and demonstrate the importance of higher-order cross-structural information in improving model performance.

Figures (6)

• Figure 1: Higher-order patterns of temporal and spatial perspectives. (A) Higher-order patterns of temporal perspective. (B) Higher-order patterns of spatial perspective.
• Figure 2: Overview of High-TS. (A) The whole framework of High-TS. Contrastive learning is constructed through multiscale embedding and TDL-based representation, followed by the output layer to obtain the final feature. (B) Multiscale embedding process. The combination of segmentation and encoder at multiple scales captures temporal embedding. (C) The TDL-based representation process. The passing of message between simplexes depicts the spatial structure.
• Figure 3: The formation process of multiscale sequence and simplicial complex. (A) The formation process of multiscale sequence. (B) The formation process of simplicial complex.
• Figure 4: Ablation study of the effect of different components in High-TS on 12 datasets.
• Figure 5: Results of grid search on the number of vertices and the dimension of latent representation on 12 datasets.