Unify and Anchor: A Context-Aware Transformer for Cross-Domain Time Series Forecasting

TL;DR

This work addresses cross-domain time series forecasting by tackling temporal pattern complexity and semantic misalignment through the Unify and Anchor paradigm. It introduces ContexTST, a Transformer variant that unifies frequency-domain representations via a Time Series Coordinator and grounds domain adaptation with contextual domain anchors through a Context-Aware Transformer and a Context-Informed MoE. The approach yields strong in-domain and zero-shot transfer performance across diverse datasets, with ablations confirming the crucial roles of frequency decomposition, contextual anchoring, and specialized experts. This framework enables more robust cross-domain forecasting with improved generalization, offering practical benefits for real-world time series applications without extensive retraining on new domains.

Abstract

The rise of foundation models has revolutionized natural language processing and computer vision, yet their best practices to time series forecasting remains underexplored. Existing time series foundation models often adopt methodologies from these fields without addressing the unique characteristics of time series data. In this paper, we identify two key challenges in cross-domain time series forecasting: the complexity of temporal patterns and semantic misalignment. To tackle these issues, we propose the ``Unify and Anchor" transfer paradigm, which disentangles frequency components for a unified perspective and incorporates external context as domain anchors for guided adaptation. Based on this framework, we introduce ContexTST, a Transformer-based model that employs a time series coordinator for structured representation and the Transformer blocks with a context-informed mixture-of-experts mechanism for effective cross-domain generalization. Extensive experiments demonstrate that ContexTST advances state-of-the-art forecasting performance while achieving strong zero-shot transferability across diverse domains.

Figures (12)

• Figure 1: The Unify and Anchor cross-domain transfer paradigm. (a) An FFT-based coordinator decomposes time series into frequency bases, establishing a unified representation that captures shared structures across domains. (b) Domain-specific contexts serve as anchors, guiding adaptation by constraining the prediction search space to align with domain characteristics.
• Figure 2: Overview of the proposed ContexTST framework. The architecture begins with the (a) Time Series Coordinator, which decomposes raw time series into orthogonal frequency components for structured representation. To complement temporal features, (b) Context Generation produces global- and variable-level textual descriptions, enriching the time series with external knowledge. Furthermore, the (c) Context-Informed MoE mechanism integrates contextual information, enabling effective generalization across domains.
• Figure 3: One-to-one zero-shot forecasting. We set ETTh1 and Electricity as the source datasets for model training and transfer ContexTST to multiple target domains. The look-back window is set to 96 for all models. Find the full results in Appendix \ref{['sec:ap_cross_domain']}
• Figure 4: Comparison with foundational time series models on diverse prediction horizons. The input sequence length is set to 96 for all models. For each foundation model, we exclude the evaluation results on its pre-trained datasets, and ContexTST is pre-trained on Electricity datasets then zero-shot inference in 4 target domains.
• Figure 5: The ablation studies about our proposed coordinator, context, and CI-MoE modules in cross-domain transfer scenario. We report here 4 ablation experiments on challenging domain transfer tasks, while for additional cross-domain and in-domain results please refer to appendix Figure \ref{['fig:ablation_studies']} and \ref{['fig:ap_cross_ablation_studies']}.