A Wave is Worth 100 Words: Investigating Cross-Domain Transferability in Time Series

TL;DR

The paper tackles cross-domain transfer in time series by introducing WQ4TS, a framework that maps heterogeneous domains into a shared spectral latent space via a Wave Quantize Module and a wavebook-based tokenization. By training a single encoder-only Transformer across multi-domain data and performing principled cross-domain pre-training with adaptive losses, the approach enables zero- and few-shot migration to new domains without altering backbone architectures. Empirical results across forecasting, imputation, and classification tasks show strong improvements, with WQ4TS achieving SOTA performance in many settings and demonstrating robust cross-domain transfer and data efficiency. The work lays groundwork for a general, scalable time series foundation model by linking spectral representations, interpretable tokens, and domain-agnostic training objectives, with potential impact on real-world deployment under diverse data regimes.

Abstract

Time series analysis is a fundamental data mining task that supervised training methods based on empirical risk minimization have proven their effectiveness on specific tasks and datasets. However, the acquisition of well-annotated data is costly and a large amount of unlabeled series data is under-utilized. Due to distributional shifts across various domains and different patterns of interest across multiple tasks. The problem of cross-domain multi-task migration of time series remains a significant challenge. To address these problems, this paper proposes a novel cross-domain pretraining method based on Wave Quantization (termed as WQ4TS), which can be combined with any advanced time series model and applied to multiple downstream tasks. Specifically, we transfer the time series data from different domains into a common spectral latent space, and enable the model to learn the temporal pattern knowledge of different domains directly from the common space and utilize it for the inference of downstream tasks, thereby mitigating the challenge of heterogeneous cross-domains migration. The establishment of spectral latent space brings at least three benefits, cross-domain migration capability thus adapting to zero- and few-shot scenarios without relying on priori knowledge of the dataset, general compatible cross-domain migration framework without changing the existing model structure, and robust modeling capability thus achieving SOTA results in multiple downstream tasks. To demonstrate the effectiveness of the proposed approach, we conduct extensive experiments including three important tasks: forecasting, imputation, and classification. And three common real-world data scenarios are simulated: full-data, few-shot, and zero-shot. The proposed WQ4TS achieves the best performance on 87.5% of all tasks, and the average improvement of the metrics on all the tasks is up to 34.7%.

Figures (2)

• Figure 1: Illustration of the proposed WQ4TS architecture. ① The proposed Wave Quantize Module was utilized to establish the common spectral latent space. ② Based on proposed Tokenization Strategy, the raw series is bijectively projected to the common spectral latent space. The distance between multiple TS domains is effectively reduced, which activates the generalization and migration capabilities of the model. Meanwhile, the generated tokens as fluctuation pattern similarities contain sufficient semantic information ③ Subsequently, the TS pattern knowledge in common spectral latent space will be utilized for multiple downstream tasks, by Cross-domain Pre-training. ④ Due to the Effective feature programming of the embedding technique, we design full parameters Fine-tuning for Domain migration. ⑤ We design the encoder-only transformer structure as the backbone, which contains a stack of EncoderLayers and OutputLayer.
• Figure 2: Model comparison in classification. The results are averaged from 35 subsets of UCR. The proposed WQ4TS achieves the best performance on the classification task under the full-data setting.

Theorems & Definitions (8)

• Definition 1
• Definition 2
• proof : Proof of Proposition \ref{['prop:1']}
• Definition 3
• proof : Proof of Proposition \ref{['prop:2']}
• proof : Proof of Proposition \ref{['prop:1']}
• Lemma B.1
• proof : Proof of Proposition \ref{['prop:2']}