FredNormer: Frequency Domain Normalization for Non-stationary Time Series Forecasting

TL;DR

It is proved that the current normalization methods that operate in the time domain uniformly scale non-zero frequencies, and thus, they struggle to determine components that contribute to more robust forecasting, and proposes FredNormer, which observes datasets from a frequency perspective and adaptively up-weights the key frequency components.

Abstract

Recent normalization-based methods have shown great success in tackling the distribution shift issue, facilitating non-stationary time series forecasting. Since these methods operate in the time domain, they may fail to fully capture the dynamic patterns that are more apparent in the frequency domain, leading to suboptimal results. This paper first theoretically analyzes how normalization methods affect frequency components. We prove that the current normalization methods that operate in the time domain uniformly scale non-zero frequencies, and thus, they struggle to determine components that contribute to more robust forecasting. Therefore, we propose FredNormer, which observes datasets from a frequency perspective and adaptively up-weights the key frequency components. To this end, FredNormer consists of two components: a statistical metric that normalizes the input samples based on their frequency stability and a learnable weighting layer that adjusts stability and introduces sample-specific variations. Notably, FredNormer is a plug-and-play module, which does not compromise the efficiency compared to existing normalization methods. Extensive experiments show that FredNormer improves the averaged MSE of backbone forecasting models by 33.3% and 55.3% on the ETTm2 dataset. Compared to the baseline normalization methods, FredNormer achieves 18 top-1 results and 6 top-2 results out of 28 settings.

Figures (5)

• Figure 1: How does z-score normalization affect the frequency amplitudes? (a) Normalization in the time domain compresses the variability of the data, leading to a more compact distribution. (b) The amplitudes of non-zero frequencies are just uniformly scaled after normalization.
• Figure 2: (a) Frequency variations across a sequence of time series samples. The red bar denotes the unstable frequency components, while stable ones are in gray. (b) An overview of $\texttt{FredNormer}$.
• Figure 3: Visualization of input sequences before and after applying $\texttt{FredNormer}$ on the Traffic, ETTh1, and ETTh2 datasets. The green line shows the input data, the blue line represents the forecasting target, and the orange line illustrates the input data generated by $\texttt{FredNormer}$. The red line represents the frequency stability measure of each dataset.
• Figure 4: Comparison of running times (s/epoch) between $\texttt{FredNormer}$ and SAN on DLinear and PatchTST. Forecasting lengths $H \in \{96, 720\}$ for all datasets and input sequence length $L = 96$.
• Figure : Frequency Stability Measure

Theorems & Definitions (7)

• Definition 1
• Definition 2
• Definition 3
• Lemma 1
• Theorem 1
• Lemma 1
• Theorem 1