Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift

TL;DR

This work tackles distribution shift in non-stationary multivariate time series forecasting by introducing JointPGM, a probabilistic graphical model with a dual-encoder neural framework that jointly models intra- and inter-series correlations and the time-variant transition between inputs and outputs. The Time Factor Encoder (TFE) learns dynamic time factors via Fourier features, while the Independence-based Series Encoder (ISE) comprises intra-series and inter-series learners that produce complementary latent representations, fused into a latent state used for prediction and reconstruction. A dynamic inference module links these latent variables to time factors, enabling discriminative sensitivity to non-stationary environments, with a decomposed objective built from three KL-based terms to enforce consistency and informative representations. Empirical results across six non-stationary MTS datasets show state-of-the-art performance and robust efficiency, with strong improvements over both general deep models and normalization-based baselines, highlighting the practical impact for robust forecasting in evolving real-world settings.

Abstract

The non-stationary nature of real-world Multivariate Time Series (MTS) data presents forecasting models with a formidable challenge of the time-variant distribution of time series, referred to as distribution shift. Existing studies on the distribution shift mostly adhere to adaptive normalization techniques for alleviating temporal mean and covariance shifts or time-variant modeling for capturing temporal shifts. Despite improving model generalization, these normalization-based methods often assume a time-invariant transition between outputs and inputs but disregard specific intra-/inter-series correlations, while time-variant models overlook the intrinsic causes of the distribution shift. This limits model expressiveness and interpretability of tackling the distribution shift for MTS forecasting. To mitigate such a dilemma, we present a unified Probabilistic Graphical Model to Jointly capturing intra-/inter-series correlations and modeling the time-variant transitional distribution, and instantiate a neural framework called JointPGM for non-stationary MTS forecasting. Specifically, JointPGM first employs multiple Fourier basis functions to learn dynamic time factors and designs two distinct learners: intra-series and inter-series learners. The intra-series learner effectively captures temporal dynamics by utilizing temporal gates, while the inter-series learner explicitly models spatial dynamics through multi-hop propagation, incorporating Gumbel-softmax sampling. These two types of series dynamics are subsequently fused into a latent variable, which is inversely employed to infer time factors, generate final prediction, and perform reconstruction. We validate the effectiveness and efficiency of JointPGM through extensive experiments on six highly non-stationary MTS datasets, achieving state-of-the-art forecasting performance of MTS forecasting.

Figures (11)

• Figure 1: Comparison of the graphical representations at time step $t$ for non-stationary MTS forecasting tasks: (a) normalization-based methods, (b) time-variant models, and (c) our proposed JointPGM. Grey circles denote observable input-output variables, while white circles denote intermediate-generated latent variables.
• Figure 2: (a) The overview of the proposed JointPGM framework featuring dual encoders. Given $N$ time series and $L+H$ time steps as input, JointPGM outputs multivariate predictions $\hat{\bm{X}}_{t:t+H}$ and reconstructions $\hat{\bm{X}}_{t-L:t}$. (b) TFE is applied to represent dynamic time factors which can reflect non-stationary environments. (c) ISE captures the intrinsic correlations within each series and among different series using two distinct learners: intra-series learner (left part of ISE) and inter-series learner (right part of ISE), organized in a consecutive fashion. (d) DI aims to reversely infer time factors from time series in latent space, designed to be dynamic to render these factors more discriminative and sensitive to non-stationarity.
• Figure 3: Evaluation on model performance with different lookback length $L$ with the range ($24 \sim 720$), horizon length $H$ with the range ($192 \sim 720$), trade-off parameter $\alpha$ ($0.1 \sim 1$) and depth of propagation $K$ ($1 \sim 6$).
• Figure 4: Model efficiency comparison under the setting of $L/H=96/192$ of Exchange (left) and Electricity (right).
• Figure 5: t-SNE visualization of series representation $\tilde{\bm{H}}_{t-L:t}$ on the Electricity test set.

Theorems & Definitions (4)

• Definition 1: Intra-series Transitional Shift
• Definition 2: Inter-series Transitional Shift
• Remark 1
• Proposition 1