UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting

TL;DR

UniTime proposes a unified cross-domain time series forecasting model that jointly handles heterogeneous domains by coupling time-series data with human-provided domain instructions via a Language-TS Transformer. It employs a Time Series Tokenizer to create tokens, a Language-TS Transformer to align text and time-series modalities in a shared latent space, and a decoder that uses padding and a lightweightTransformer to produce flexible forecast lengths, all trained with a dual objective of forecasting and history reconstruction. The model addresses three core challenges—varying data characteristics, domain confusion, and convergence-speed imbalance—through channel-independence, explicit domain instructions, and masking. Empirically, UniTime achieves state-of-the-art results across eight benchmarks, demonstrates strong zero-shot transferability, and shows robustness to instruction variation and language-model choices, suggesting a foundation-model-like capability for general time series forecasting with practical impact on scalable, multi-domain web systems.

Abstract

Multivariate time series forecasting plays a pivotal role in contemporary web technologies. In contrast to conventional methods that involve creating dedicated models for specific time series application domains, this research advocates for a unified model paradigm that transcends domain boundaries. However, learning an effective cross-domain model presents the following challenges. First, various domains exhibit disparities in data characteristics, e.g., the number of variables, posing hurdles for existing models that impose inflexible constraints on these factors. Second, the model may encounter difficulties in distinguishing data from various domains, leading to suboptimal performance in our assessments. Third, the diverse convergence rates of time series domains can also result in compromised empirical performance. To address these issues, we propose UniTime for effective cross-domain time series learning. Concretely, UniTime can flexibly adapt to data with varying characteristics. It also uses domain instructions and a Language-TS Transformer to offer identification information and align two modalities. In addition, UniTime employs masking to alleviate domain convergence speed imbalance issues. Our extensive experiments demonstrate the effectiveness of UniTime in advancing state-of-the-art forecasting performance and zero-shot transferability.

Figures (6)

• Figure 1: (a) Specialized models are separately trained on time series domains with notable distribution differences. For instance, weather time series constantly fluctuate due to the chaotic influence of natural factors, while economic data, such as exchange rates, tends to remain relatively stable. Disease data, like seasonal cold patterns, typically demonstrate periodicity over extended time periods. (b) The proposed cross-domain learning approach handles time series data from distinct domains and utilizes natural language as domain instructions to provide domain-specific information.
• Figure 2: UniTime overview from the perspective of a univariate time series.
• Figure 3: Visualization of the validation loss during model training. The x-axis denotes the training epoch number.
• Figure 4: T-SNE visualization of the hidden representations.
• Figure 5: Effects of mask ratio. The y-axis is the average test MSE over four predictive lengths.