CrossLinear: Plug-and-Play Cross-Correlation Embedding for Time Series Forecasting with Exogenous Variables

TL;DR

CrossLinear introduces a lightweight plug-and-play cross-correlation embedding for time series forecasting with exogenous variables, focusing on time-invariant, direct dependencies between endogenous and exogenous factors. The architecture combines a 1D convolution-based cross-correlation module with patch embedding and a global forecasting head, reinforced by RevIN normalization, to capture both short- and long-term temporal dynamics. Empirical results across 12 real-world datasets show CrossLinear achieving state-of-the-art performance in many-to-one and strong gains in multivariate settings, with ablations confirming the effectiveness of the integration strategy and the modular plug-in nature. The approach offers a favorable efficiency profile and broad generalizability, making it practical for deployment and adaptable to a range of forecasting tasks and domains.

Abstract

Time series forecasting with exogenous variables is a critical emerging paradigm that presents unique challenges in modeling dependencies between variables. Traditional models often struggle to differentiate between endogenous and exogenous variables, leading to inefficiencies and overfitting. In this paper, we introduce CrossLinear, a novel Linear-based forecasting model that addresses these challenges by incorporating a plug-and-play cross-correlation embedding module. This lightweight module captures the dependencies between variables with minimal computational cost and seamlessly integrates into existing neural networks. Specifically, it captures time-invariant and direct variable dependencies while disregarding time-varying or indirect dependencies, thereby mitigating the risk of overfitting in dependency modeling and contributing to consistent performance improvements. Furthermore, CrossLinear employs patch-wise processing and a global linear head to effectively capture both short-term and long-term temporal dependencies, further improving its forecasting precision. Extensive experiments on 12 real-world datasets demonstrate that CrossLinear achieves superior performance in both short-term and long-term forecasting tasks. The ablation study underscores the effectiveness of the cross-correlation embedding module. Additionally, the generalizability of this module makes it a valuable plug-in for various forecasting tasks across different domains. Codes are available at https://github.com/mumiao2000/CrossLinear.

Figures (6)

• Figure 1: Different forecasting paradigms. Here, $\mathbf{X}_{i}$ represents the $i^\mathrm{th}$ variable, $\mathbf{X}^{endo}_{1}$ is the endogenous variable, and $\mathbf{X}^{exo}_{i}$ denotes the $i^\mathrm{th}$ exogenous variable. (a) Univariate forecasting. (b) Multivariate forecasting. (c) Forecasting with exogenous variables.
• Figure 2: Architecture of CrossLinear. (a) The cross-correlation embedding module captures variable dependencies. (b) The patch embedding module captures short-term temporal dependencies. (c) The linear head is responsible for capturing long-term patterns.
• Figure 3: Hyperparameter sensitivity with respect to $T$, $\alpha$, and $\beta$. For $T$, $S=96$; for $\alpha$ and $\beta$, $T=96$ and $S=96$. More results can be found in Appendix \ref{['sec: hyperparameter sensitivity']}.
• Figure 4: Model anlysis. (a) Variate-wise correlation analysis. (b) Model efficiency analysis.
• Figure 5: Hyperparameter sensitivity with respect to patch size ($p$), convolution kernel size, and the number of hidden state units. Here, $T=96$ and $S=96$.