Lightweight Channel-wise Dynamic Fusion Model: Non-stationary Time Series Forecasting via Entropy Analysis

TL;DR

A novel lightweight lightweight time series forecasting model, which selectively and dynamically recovers intrinsic non-stationarity of the original series, while keeping the predictability of normalized series, is proposed.

Abstract

Non-stationarity is an intrinsic property of real-world time series and plays a crucial role in time series forecasting. Previous studies primarily adopt instance normalization to attenuate the non-stationarity of original series for better predictability. However, instance normalization that directly removes the inherent non-stationarity can lead to three issues: (1) disrupting global temporal dependencies, (2) ignoring channel-specific differences, and (3) producing over-smoothed predictions. To address these issues, we theoretically demonstrate that variance can be a valid and interpretable proxy for quantifying non-stationarity of time series. Based on the analysis, we propose a novel lightweight \textit{C}hannel-wise \textit{D}ynamic \textit{F}usion \textit{M}odel (\textit{CDFM}), which selectively and dynamically recovers intrinsic non-stationarity of the original series, while keeping the predictability of normalized series. First, we design a Dual-Predictor Module, which involves two branches: a Time Stationary Predictor for capturing stable patterns and a Time Non-stationary Predictor for modeling global dynamics patterns. Second, we propose a Fusion Weight Learner to dynamically characterize the intrinsic non-stationary information across different samples based on variance. Finally, we introduce a Channel Selector to selectively recover non-stationary information from specific channels by evaluating their non-stationarity, similarity, and distribution consistency, enabling the model to capture relevant dynamic features and avoid overfitting. Comprehensive experiments on seven time series datasets demonstrate the superiority and generalization capabilities of CDFM.

Figures (11)

• Figure 1: Issues caused by instance normalization. (a) Instance normalization changes the original temporal dependencies. (b) Different channels exhibit unique non-stationarity, such as seasonal and trend channels. (c) The over-smoothing problem caused by instance normalization. These predictions are obtained by DLinear zeng2023transformers.
• Figure 2: Normalization mitigates the distribution discrepancy between different samples.
• Figure 3: The relationship between variance and information entropy for the 'HUFL' and 'OT' channels in ETTm1.
• Figure 4: The architecture of CDFM consists of three key components: (1) The Dual-Predictor Module processes the input $\mathcal{X}$ through two complementary branches to get the stationary prediction $\hat{\mathcal{Y}_s}$ and the non-stationary prediction $\hat{\mathcal{Y}_{ns}}$, respectively; (2) The Fusion Weight Learner dynamically learns fusion weight $W$ of samples based on the historical standard deviation $\sigma_{t,\mathbf{x}}$ and the predicted horizon standard deviation $\hat{\sigma}_{t,\mathbf{y}}$; (3) The Channel Selector selects the top-k channels by evaluating their non-stationarity, similarity, and distributional consistency, and then recovers non-stationary information of these channels to obtain the final prediction $\hat{\mathcal{Y}}$.
• Figure 5: The dynamic fusion weight of input-96-predict-192 on the ETTh2 dataset. The highly non-stationary samples have large weights.